Tag Archives: back-end

Self test error when springboot accesses es and starts

1、 Background

Recently, I was working on a project to connect two es clusters, so I initialized two resthighlevelclient instances esclient and esclient1

package com.xxx.common.config;

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class EsConfig {

    @Value("${es.url}")
    private String url;

    @Value("${es.port}")
    private Integer port;

    @Value("${es.username}")
    private String username;

    @Value("${es.password}")
    private String password;

    @Value("${es.connection.timeout:30000}")
    private int connctionTimeout;

    @Value("${es.socket.timeout:60000}")
    private int socketTimeout;

    @Bean
    public RestHighLevelClient esClient() {
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));

        RestClientBuilder builder = RestClient.builder(new HttpHost(url, port))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                }).setRequestConfigCallback(requestConfigBuilder -> requestConfigBuilder.setConnectTimeout(connctionTimeout)
                        .setSocketTimeout(socketTimeout));
        RestHighLevelClient client = new RestHighLevelClient(builder);
        return client;
    }
}

As a result, an error was reported during startup, as follows:

java.net.ConnectException: Connection refused
 at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:823)
 at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
 at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
 at org.springframework.boot.actuate.elasticsearch.ElasticsearchRestHealthIndicator.doHealthCheck(ElasticsearchRestHealthIndicator.java:60)
 at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82)
 at org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37)
 at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:71)
 at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:39)
 at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:99)
 at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateHealth(HealthEndpointSupport.java:110)
 at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:96)
 at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:74)
 at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:61)
 at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:65)
 at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:55)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
 at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:77)
 at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60)
 at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:121)
 at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:96)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
 at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
 at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
 at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
 at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
 at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
 at sun.rmi.transport.Transport$1.run(Transport.java:200)
 at sun.rmi.transport.Transport$1.run(Transport.java:197)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
 at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
 at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
 at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Caused by: java.net.ConnectException: Connection refused
 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715)
 at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
 at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
 at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
 at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
 at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
 ... 1 common frames omitted

  2、 Analyze the cause

Why only one client will not report an error?If two clients report an error through self-test, and the URL in the client is【 http://localhost:9200 】, with such questions, I read the source code of ES self-test once

ElasticsearchRestHealthIndicator

The main method is this

public ElasticsearchRestHealthIndicator(RestClient client) {
    super("Elasticsearch health check failed");
    this.client = client;
    this.jsonParser = JsonParserFactory.getJsonParser();
}

protected void doHealthCheck(Builder builder) throws Exception {
    Response response = this.client.performRequest(new Request("GET", "/_cluster/health/"));
    StatusLine statusLine = response.getStatusLine();
    if (statusLine.getStatusCode() != 200) {
        builder.down();
        builder.withDetail("statusCode", statusLine.getStatusCode());
        builder.withDetail("reasonPhrase", statusLine.getReasonPhrase());
    } else {
        InputStream inputStream = response.getEntity().getContent();
        Throwable var5 = null;

        try {
            this.doHealthCheck(builder, StreamUtils.copyToString(inputStream, StandardCharsets.UTF_8));
        } catch (Throwable var14) {
            var5 = var14;
            throw var14;
        } finally {
            if (inputStream != null) {
                if (var5 != null) {
                    try {
                        inputStream.close();
                    } catch (Throwable var13) {
                        var5.addSuppressed(var13);
                    }
                } else {
                    inputStream.close();
                }
            }

        }

    }
}

The reason for the error is that the URL in the client configuration is not pointed to correctly【 http://localhost:9200 】So curiosity showed me the ES auto configuration source code

ElasticsearchRestClientAutoConfiguration

ElasticsearchRestClientConfigurations

ElasticsearchRestClientProperties

RestClientBuilderCustomizer

I found something fishy in elasticsearchrestclientconfigurations, as shown in the following code

    @Bean
    @ConditionalOnMissingBean
    RestClient elasticsearchRestClient(RestClientBuilder builder, ObjectProvider<RestHighLevelClient> restHighLevelClient) {
        RestHighLevelClient client = (RestHighLevelClient)restHighLevelClient.getIfUnique();
        return client != null ?client.getLowLevelClient() : builder.build();
    }

This code indicates that if there are multiple resthighlevelclient instances in the current application, restclientbuilder will be selected. If there is only one instance, the default one will be selected. This also explains the default configuration of elasticsearchrestclientproperties if there are multiple clients without an error

private List<String> uris = new ArrayList(Collections.singletonList("http://localhost:9200"));
private String username;
private String password;
private Duration connectionTimeout = Duration.ofSeconds(1L);
private Duration readTimeout = Duration.ofSeconds(30L);

3、 Solution

1. If there are multiple es, remove the self-test (not recommended)

management.health.elasticsearch.enabled=false

2. Configure the default restclientbuilder to self check the default es

The first method: keep the name the same as the default resthighlevelclient

@Bean(name = "restHighLevelClient")
public RestHighLevelClient restHighLevelClient() {
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));

        RestClientBuilder builder = RestClient.builder(new HttpHost(url, port))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                }).setRequestConfigCallback(requestConfigBuilder -> requestConfigBuilder.setConnectTimeout(connctionTimeout)
                        .setSocketTimeout(socketTimeout));

        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(builder);
        return restHighLevelClient;
}

The second method: set @ primary as the default

@Bean
@Primary
public RestHighLevelClient esClient() {
    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));

    RestClientBuilder builder = RestClient.builder(new HttpHost(url, port))
            .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                }
            }).setRequestConfigCallback(requestConfigBuilder -> requestConfigBuilder.setConnectTimeout(connctionTimeout)
                    .setSocketTimeout(socketTimeout));
    RestHighLevelClient client = new RestHighLevelClient(builder);
    return client;
}

3. Repeat the ES self-test procedure (allow multiple es clusters to perform self-test)

What I’m interested in is to expand and enable the two clusters to implement health check

How to Solve JVM Common Errors: outofmemoryerror & stackoverflowerror

OutOfMemoryError

Error cause: java.lang.outofmemoryerror: Java heap space heap memory overflow
solution: adjust the size of the heap memory

// -Xms1m -Xmx10m -XX:+PrintGCDetails
		List<Object> listObject = new ArrayList<>();
		for (int i = 0; i < 10; i++) {
			System.out.println("i:" + i);
			Byte[] bytes = new Byte[1 * 1024 * 1024];
			listObject.add(bytes);
		}
		System.out.println("Added successfully...");

StackOverflowError

Cause of error: java.lang.stackoverflowerror is expressed as stack overflow, which generally occurs in recursive calls
solution: set the maximum thread call depth, which is 1m by default

//-Xss5m Set the maximum call depth
public class StackTest {
	private static int count;
	public static void count(){
		try {
			count++;
			count(); 
		} catch (Throwable e) {
			System.out.println("the maximum depth:"+count);
			e.printStackTrace();
		}
	}
	public static void main(String[] args) {
			 count();
	}
}

[Solved] Initialization of anonymous inner class member variable causes java.lang.stackoverflowerror

Project scenario:

An abstract class A in Java needs to initialize a member variable of the same type anonymously,

public class Main {
  public static void main(String[] args) {
    new B();
  }

}

abstract class A {
   A a = new A() {//Member variables of the same type
     @Override
     void do_sth() {
       System.out.println("do nothing");
     }
   };
  abstract void do_sth();
}

class B extends A{

  @Override
  void do_sth() {
	System.out.println("doing B");
  }
}

Problem Description:

Java.lang.stackoverflowerror directly overflowed the stack

Cause analysis:

When creating object B, the anonymous inner class in object a is also created, and the inner class creates its own inner class, resulting in infinite recursion.

Solution:

Try not to use anonymous inner classes as member variables. If you want to use them, be sure to pay attention to whether they contain the possibility of infinite recursion.

Could not write JSON: write javaBean error, fastjson version x.x.x, class

Could not write JSON: write javaBean error, fastjson version x.x.x, class

Possible causes of error reporting (welcome to add)

1. Serialization problem. The reason is that some fields in the bean object cannot be serialized. For example, fields of multipart type cannot be serialized. You can not serialize annotations through @ jsonfailed (serialize = false)

2. When converting JSON to object, there are some syntax problems in the attribute get/set, such as null pointer, type conversion, divisor and dividend in division are 0, etc

3. Syntax problem in get/set when database query returns (as above)

How to Solve Ubuntu20.04 Compile Error：error while loading shared libraries: libncurses.so.5

Project scenario:

Ubuntu 20.04 compilation error: error while loading shared libraries: libncurses. So. 5: cannot open shared object file: no such file or directory

Cause analysis:

libncurses5 file is missing 

Solution:

install libncurses5 to solve the problem. The commands are as follows

$      sudo apt install libncurses5

Error creating bean with name ‘helloController‘: Injection of autowired dependencies failed；

Error content:
when using the automatic injection of springboot, the configuration file in yaml format is used for injection, but the error is always reported and cannot be injected.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘helloController’: Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder ‘name’ in value “${name}”

I searched many posts saying that I should pay attention to the application scanning my folder and corresponding subfolders. Many people step on the pit in this place, but my problem is not here
code

@RestController
public class HelloController {

    @Value(value = "${name}")
    private String name;
    
    @RequestMapping("/hello")
    public String hello(){
        System.out.println(name);
        return "hello Spring Boot !";
    }
}

Add the annotation to read the configuration file on hellocontroller

@PropertySource(value = "classpath:application.yaml")

That solved the problem

Failed to get the resources path in the jar package, reporting an error null pointer

Background

The springboot project calls the method of a local jar package and needs the file path under the project resources. The file is stored under resources.

Trial and error process

1. Original code:

First use classloader.getsystemresource ("file name") , and the local operation is normal. However, an error null pointer is reported after the jar package is typed.

2. First modification:

Most people on the Internet say that they use this. Getclass(). Getclassloader(). Getresource() , but they still can’t. the two principles are the same.

3. Second modification:

Another method is to use the file stream to obtain the file, this. Getclass(). Getclassloader(). Getresourceasstream() , but this is to obtain the file directly. It needs to be the file path, but it can’t.

4. Finally:

Use string filepath = system. Getproperty ("user. Dir") + "\ \ file name" , get the path of the jar package, add the file name to get the file address, but you need to put the file in the same file directory as the jar package.

dot
system. Getproperty (“user. Dir”) + “\ double slash in file name” is in Windows environment. If it needs to be changed to “/” in Linux environment.

(at present, we haven’t found a way not to put the documents outside. There is a solution. I hope the bosses will give us their advice!)

[Solved] Error: Another program is already listening on a port that one of our HTTP servers is configured to

Question

    1. Step 1 run
supervisord  -c /etc/supervisor/supervisord.conf

Error

Error: Another program is already listening on a port that one of our HTTP servers is configured to

Solution:

ps -ef | grep supervisord

Turn off the 8079 process

kill -s SIGTERM 8079

Solution results

supervisord  -c /etc/supervisor/supervisord.conf

No error is the best case, indicating that the process is started successfully

ERROR: Pool overlaps with other one on this address space

Error: pool overlaps with other one on this address space is sometimes reported when creating networks in docker. This is because the target network segment already exists. Usually, this occurs when the network to be created manually specifies the subnet network segment address.

At this time, there are two options: one is to check whether the container of the existing network segment is in use. If it is no longer used, you can delete the network card corresponding to the network segment, and the other is to modify the address of the target subnet network segment.

Docker network LS
View docker network card

Docker network RM network card ID
delete docker network card

Docker network inspect network card ID
view the details of the docker network card