Author Archives: Robins

[Solved] Kafka Error: is/are not present and missingTopicsFatal is true

 

1. Error message

org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is java.lang.IllegalStateException: Topic(s) [350002000000000042] is/are not present and missingTopicsFatal is true
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185)
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:893)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204)
	at com.iwhalecloud.oss.res.search.ApplicationMain.main(ApplicationMain.java:21)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.lang.IllegalStateException: Topic(s) [350002000000000042] is/are not present and missingTopicsFatal is true
	at org.springframework.kafka.listener.AbstractMessageListenerContainer.checkTopics(AbstractMessageListenerContainer.java:318)
	at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:136)
	at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:292)
	at org.springframework.kafka.config.KafkaListenerEndpointRegistry.startIfNecessary(KafkaListenerEndpointRegistry.java:311)
	at org.springframework.kafka.config.KafkaListenerEndpointRegistry.start(KafkaListenerEndpointRegistry.java:255)
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182)
	... 22 common frames omitted

2. Cause analysis

The above error is caused because the monitored Kafka topic does not exist.

 

3. Solutions

1. Enable Kafka to automatically create topics

2. Manually go to Kafka to create the required topic.

3. The function of Kafka monitoring and checking topic is turned off in the code, as shown in the following code:

spring.kafka.listener.missing-topics-fatal=false

 

How to Solve mybatis-plus Paging Plug-in PaginationInnerInterceptor error

Questions

mybatis-plus using PaginationInnerInterceptor paging plugin, when calling the paging query method (****Service.page(new Page(param.getPage(),param.getPageSize()),queryWrapper )) reports the following error:

 2022-07-22 17:07:20.699 [TID: N/A]  WARN 12444 --- [io-18080-exec-1] c.b.m.e.p.i.PaginationInnerInterceptor   : optimize this sql to a count sql has exception, sql:"sql语句略", exception:
net.sf.jsqlparser.parser.ParseException: Encountered unexpected token: "," ","
    at line 1, column 191.

Was expecting one of:

    "&"
    "::"
    ";"
    "<<"
    ">>"
    "ACTION"
    "ACTIVE"
    "ALGORITHM"
    "ARCHIVE"
    "ARRAY" 
    ***略****

reason

The paging plugin will default to sql optimization when processing count, and will throw an exception if optimization fails. The sql that cannot be optimized will be downgraded to the non-optimized count method.
The code summary is as follows.
where Select select = (Select) CCJSqlParserUtil.parse(sql); This line of code reports error:

    protected String autoCountSql(IPage<?> page, String sql) {
        if (!page.optimizeCountSql()) {
            return lowLevelCountSql(sql);
        }
        try {
            Select select = (Select) CCJSqlParserUtil.parse(sql);
            **************Optimization Logic*********************
            return select.toString();
        } catch (JSQLParserException e) {
            // Unable to optimize the use of the original SQL
            logger.warn("optimize this sql to a count sql has exception, sql:\"" + sql + "\", exception:\n" + e.getCause());
        } catch (Exception e) {
            logger.warn("optimize this sql to a count sql has error, sql:\"" + sql + "\", exception:\n" + e);
        }
        return lowLevelCountSql(sql);
    }

Solution:

Add the following codes before your codes:

if (!page.optimizeCountSql()) {
return lowLevelCountSql(sql);
}

just set optimizeCountSql to false, as follows

Page page = new Page(param.getPage(),param.getPageSize()),queryWrapper)
page.setOptimizeCountSql(false);

You can also redefine a class to inherit Page, and set optimizeCountSql default to =false

 

How to Solve Spring Boot Maven Build Error

Error 1:

[ERROR]Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on project motherBuyBoot: There are test failures.
[ERROR] Please refer to D:\web\motherbuy\target\surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.

Solution:

the pom.xml file has the wrong version of Junit, or for some other reason, the junit dependency is commented out and the compilation passes.

 

Error 2:

Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project motherBuyBoot: Compilation failure
[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?

Solution:

Method: Compile compile environment problems, that are not configured; see the screenshot below to solve, do not forget to update maven.

[Solved] fontconfig cross-compilation Error: PRI_CHAR_WIDTH_STRONG 

make report an error as below:
“fontconfig-2.12.1/src/fcmatch.c:324:63: error: ‘PRI_CHAR_WIDTH_STRONG’ undeclared here (not in a function); did you mean ‘PRI_WIDTH_STRONG’?”

"fontconfig-2.12.1/src/fcmatch.c:324:63: error: ‘PRI_CHAR_WIDTH_STRONG' undeclared here (not in a function); did you mean ‘PRI_WIDTH_STRONG’?"

 

Solution:
Enter the source code directory. My path is
1.
fontconfig-2.12.1/fontconfig/fontconfig.h
Found #define FC_CHAR_WIDTH “charwidth” /* Int / deleted
Add #define FC_CHARWIDTH “charwidth” / Int */
#define FC_CHAR_WIDTH FC_CHARWIDTH
2.
fontconfig-2.12.1/src/fcobjs.h
Find FC_OBJECT (CHAR_WIDTH, FcTypeInteger, NULL) and delete it.
Add FC_OBJECT (CHARWIDTH, FcTypeInteger, NULL)
3.
fontconfig-2.12.1/src/fcobjshash.gperf
Find “CHARWIDTH”, FC_CHAR_WIDTH_OBJECT and delete it.
Add “charwidth”,FC_CHARWIDTH_OBJECT
4.
fontconfig-2.12.1/src/fcobjshash.h
Find {(int)(long)&((struct FcObjectTypeNamePool_t *)0)->FcObjectTypeNamePool_str45,FC_CHAR_WIDTH_OBJECT}, delete
add {(int)(long)&((struct FcObjectTypeNamePool_t *)0)->FcObjectTypeNamePool_str45,FC_CHARWIDTH_OBJECT},
End, make && make install successful
is to modify and replace a few macro definitions, you can happily edit cairo!
Although the high version of fontconfig does not have this error, but the cross-compiler reported that the other failed to solve

 

How to Solve args = parser.parse_args() error

args = parser.parse_args() error

Problem description

args = parser.parse_args() reports an error

import argparse

parser = argparse.ArgumentParser(prog='top', description='Show top lines from each file')
parser.add_argument('filenames', nargs='+')
parser.add_argument('-l', '--lines', type=int, default=10)
# args = parser.parse_args()
args, unknown = parser.parse_known_args() # Modified
print(args)

Solution:
Modify to args, unknown = parser.parse_known_args()

[Solved] Project Error: Could not found sun.misc.BASE64Encoder

1. Questions

The project reported an error: could not found sun.misc.BASE64Encoder

2. Reason

Classes such as sun.misc.BASE64Encoder are not part of the JDK standard library, but the class is included in the JDK and can be used directly.

3. Solution

Replace sun.misc.BASE64Encoder with java.util.Base64

Example:

//import sun.misc.BASE64Encoder;
import java.util.Base64;//how to solve could not found sun.misc.BASE64Encoder error
...
// return new BASE64Encoder().encodeBuffer(bytes).trim().replaceAll("\r|\n", "");
return Base64.getEncoder().encodeToString(bytes);

[Solved] Compile Error: undefined reference to `google::FlagRegisterer::FlagRegisterer

1. Problem description

Compilation error

CMakeFiles/run_mapping_offline.dir/run_mapping_offline.cc.o: In function `__static_initialization_and_destruction_0':
/home/wong/workspace_demo/packagetest_faster_lio_ws/src/***/***/app/run_mapping_offline.cc:16: undefined reference to `google::FlagRegisterer::FlagRegisterer<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'
/home/wong/workspace_demo/packagetest_faster_lio_ws/src/***/***/app/run_mapping_offline.cc:17: undefined reference to `google::FlagRegisterer::FlagRegisterer<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'
/home/wong/workspace_demo/***/src/***/***/app/run_mapping_offline.cc:18: undefined reference to `google::FlagRegisterer::FlagRegisterer<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'
/home/wong/workspace_demo/***/src/***/***/app/run_mapping_offline.cc:19: undefined reference to `google::FlagRegisterer::FlagRegisterer<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'
collect2: error: ld returned 1 exit status
***/***/app/CMakeFiles/run_mapping_offline.dir/build.make:678: recipe for target '/home/wong/workspace_demo/***/devel/lib/faster_lio/run_mapping_offline' failed

2. Solution

Add the following line in CMakeLists.txt:

find_package(gflags REQUIRED)

This is our own solution. If you have any other better solutions, please leave a comment and let me know.

[Solved] Mybatis Error: CannotFindDataSourceException: dynamic-datasource can not find primary datasource

Problem background

When using mybatis, an error is reported that the master data source cannot be found:

org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.exceptions.PersistenceException: 
### Error updating database.  Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
### The error may exist in com/lanran/transactional/dao/PaymentMapper.java (best guess)
### The error may involve com.lanran.transactional.dao.PaymentMapper.insertBatchSomeColumn
### The error occurred while executing an update
### Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource

Solution

1. Because the multi-source configuration dependency of mybatisplus is added, but the configuration file does not write the related configuration of multi-source, an error is reported

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>dynamic-datasource-spring-boot-starter</artifactId>
            <version>3.5.0</version>
        </dependency>

Method 1: if you don’t use multi-source configuration, remove this dependency
Method 2:  application set multi-source configuration

server:
  port: 40001

spring:
  datasource:
    druid:
      stat-view-servlet:
        enabled: true
    dynamic:
      # Configure the global druid parameter, please configure as needed
      druid:
        initial-size: 5
        max-active: 8
        min-idle: 3
        max-wait: 1000
        validation-query: 'select 1'
      datasource:
        master:
          username: root
          password: 123456
          url: jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=utf-8&useSSL=true&serverTimezone=UTC
          driver-class-name: com.mysql.cj.jdbc.Driver
        slave_1:
          username: root
          password: 123456
          url: jdbc:mysql://localhost:3306/test2?useUnicode=true&characterEncoding=utf-8&useSSL=true&serverTimezone=UTC
          driver-class-name: com.mysql.cj.jdbc.Driver

[Solved] Internal error occurred: failed calling webhook “validate.nginx.ingress.kubernetes.io”:

Leave a reply

Error Message:

 

 

 

 

Solution:

enter this command as following:

[root@k8s-master01 ingress]# kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io “ingress-nginx-admission” deleted
[root@k8s-master01 ingress]# kubectl apply -f ingress1.yaml
ingress.networking.k8s.io/nginx-test created
[root@k8s-master01 ingress]#

[Solved] node.js request Error: Error: unable to verify the first certificate

Leave a reply

Solution:

Install request-promise

const request = require('request-promise');
var options = {
    method: 'POST',
    uri: 'http://api.posttestserver.com/post',
    "rejectUnauthorized": false, 
    body: {
        some: 'payload'
    },
    json: true // Automatically stringifies the body to JSON
};
 
rp(options)
    .then(function (parsedBody) {
        // POST succeeded...
    })
    .catch(function (err) {
        // POST failed...
    });

The point is to add: “rejectUnauthorized”: false

[Solved] pytorch Load Error: “RuntimeError: Error(s) in loading state_dict for Sequential:”

Leave a reply

RuntimeError: Error(s) in loading state_dict for Sequential:

This error is usually related to the use of nn.DataParallel for training

It means that the string in the model’s parameter key does not match the string in the key fetched by torch.load

Therefore, we just need to modify the dict obtained by torch.load to make it match.

 

Example

When I torch.save, the string in the parameter key is automatically prepended with ‘module.’

Therefore, after torch.load, we need to remove ‘module.

The method is as follows.

model = Model()
model_para_dict_temp = torch.load('xxx.pth')
model_para_dict = {}
for key_i in model_para_dict_temp.keys():
    model_para_dict[key_i[7:]] = model_para_dict_temp[key_i]  # Delete 'module.'
del model_para_dict_temp
model.load_state_dict(model_para_dict)