When kibana is started, it fails to start all the time, and the error elasticsearch is reported service failed after enable elasticsearch security features
Solution:
1. Stop Kibana, stop ES
2. Open elasticsearch.yml file, configure disable security option xpack.security.enabled
xpack.security.enabled: false3. Restart Elasticsearch and Kibana
Error reporting information
[WARN ][o.e.x.m.e.l.LocalExporter] [es03.kpt.rongtime.aliapse5.id] unexpected error while indexing monitoring documentorg.elasticsearch.xpack.monitoring.exporter.ExportException: RemoteTransportException[[es01.kpt.rongtime.aliapse5.id][10.107.0.247:29300][indices:admin/create]]; nested: IllegalArgumentException[Validation Failed: 1: this action would add [6] total shards, but this cluster currently has [2998]/[3000] maximum shards open;];
at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$throwExportException$2(LocalBulk.java:125) ~[x-pack-monitoring-7.4.2.jar:7.4.2] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?]
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.throwExportException(LocalBulk.java:126) [x-pack-monitoring-7.4.2.jar:7.4.2]
at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$doFlush$0(LocalBulk.java:108) [x-pack-monitoring-7.4.2.jar:7.4.2]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.4.2.jar:7.4.2] at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:70) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:64) [elasticsearch-7.4.2.jar:7.4.2] at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.bulk.TransportBulkAction$BulkOperation.doRun(TransportBulkAction.java:421) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.bulk.TransportBulkAction.executeBulk(TransportBulkAction.java:551) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.bulk.TransportBulkAction$1.onFailure(TransportBulkAction.java:287) [elasticsearch-7.4.2.jar:7.4.2] at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:79) [elasticsearch-7.4.2.jar:7.4.2]
at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:50) [elasticsearch-7.4.2.jar:7.4.2]
Cause of problem
[indices:admin/create]]; nested: IllegalArgumentException[Validation Failed: 1: this action would add [6] total shards, but this cluster currently has [2998]/[3000] maximum shards open;];
For elasticsearch7 and above, only 3000 shards are allowed by default, which is caused by the insufficient number of available shards in the cluster.
Solution:
PUT /_cluster/settings
{
"transient": {
"cluster": {
"max_shards_per_node":10000
}
}
}
Question: Java lang.RuntimeException: can not run elasticsearch as root
Solution:
Elaticsearch cannot be started by root by default, so it will report Java.Lang.runtimeException: can not run elasticsearch as root.
There are two types of solutions:
1. Modify the elaticsearch configuration to allow the root user to start (not recommended)
#Add the parameter -Des.insecure.allow.root=true when executing elasticSearch, the complete command is as follows
./elasticsearch -Des.insecure.allow.root=true#Or open the elasticsearch executable file with vi, and add the following command before the variable ES_JAVA_OPTS is used
ES_JAVA_OPTS="-Des.insecure.allow.root=true"2. Create users for elaticsearch and grant corresponding permissions
adduser es
passwd es
chown -R es:es elasticsearch-6.3.2/
chmod 770 elasticsearch-6.3.2/Background
Visit kinaba in the web page http://localhost:5601 , always prompt “kibana server is not ready yet”.
Execute the following command to view kibana logs,
docker logs kibanaTips found:
Text
It is suspected that the internal IP of each container changes after the container is restarted.
1. Therefore, execute the following command to check the internal IP of elasticsearch container and find that it is kibana The ES container IP in yaml configuration file is inconsistent with the actual es container IP.
docker inspect --format '{{ .NetworkSettings.IPAddress }}' es container ID
// Check the id of es container
docker ps2. Enter kibana container and update kibana.Yaml configuration file. Execute the following command to enter and edit kibana.yaml，
docker exec -it kibana container id /bin/bash
cd config
vi kibana.ymlReplace the IP address of the selected part in the figure below with the actual es container IP address, save and exit kibana.
3. Stop kibana service, delete kibana container and restart kibana.
// 3.1 Stop the kibana service
docker stop kibana container id
// 3.2 Delete the kibana container. (Not delete the kibana image! Not delete the kibana image! Not delete the kibana image!)
docker rm -f kibana container id
// 3.3 Enable running kibana
docker run --name kibana -e ELASTICSEARCH_HOST=http://es_contaner_ip:9200 -p 5601:5601 -d kibana:7.7.0
// Note: The command 'kibana:' in 3.3 above is followed by the kibana version number. To be on the safe side, it is recommended that the version of elasticsearch and kibana remain the same.
4. Browser revisit http://localhost:5601, refresh a few more times to access kibana normally.
The environment background is docker DNMP installation of ES + PHP
Corresponding docker compose configuration and corresponding port
Port 9200 can also be accessed normally
Code
Then 500 errors are reported, including:
PHP Fatal error: Uncaught Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in /www/localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php:67
Stack trace:
#0 /www/localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(87): Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection()
#1 /www/localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(105): Elasticsearch\Transport->getConnection()
#2 /www/localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(275): Elasticsearch\Transport->performRequest('POST', '/my_index/my_ty...', Array, '{"testField":"a...', Array)
#3 /www/localhost/vendor/react/promise/src/FulfilledPromise.php(28): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array)
#4 /www/localhost/vendor/ezimuel/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise\FulfilledPromise->then(Object(Clos in /www/localhost/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php on line 67The reason is that the node cannot be found. The solution is as follows: find your own IP and specify the IP
Then the execution is OK
In the process of learning ES6, an error is reported when and running JavaScript files, which is reproduced in detail
1. Create a new file, add test es, and use idea to open the folder
2. Add package.json file: NPM init – y
3. Create a new test.js file in the folder test-es, the contents of which are as follows
let a = 1111
let b = 2222
var c = function(){
console.log(3333)
}
function ddd(){
console.log(4444)
}
function eeeeee5(){
console.log(555)
}
export {
a,
b,
c,
ddd,
eeeeee5 as eee, //The way to rename: name eeeeeeee as eee, you can output the same variable or method with different names
eeeeee5 as e5 // rename the way: name eeeeee as eee, you can output the same variable or method with a different name
}
4. Right click and click Run ‘test.JS’, the error is reported as follows:
Solution:
According to the error message, in package JSON add “type”: “module”, and then run test.JS file, no error will be reported
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'restHighLevelClient' defined in class path resource [com/esdemo/config/ElasticSearchClientConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.RestHighLevelClient]: Factory method 'restHighLevelClient' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/DeprecationHandler
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:590)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1247)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1096)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:535)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:759)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:762)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:398)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246)
at com.esdemo.EsdemoApplication.main(EsdemoApplication.java:13)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.RestHighLevelClient]: Factory method 'restHighLevelClient' threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/DeprecationHandler
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:582)
... 18 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/DeprecationHandler
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.elasticsearch.client.RestHighLevelClient.<clinit>(RestHighLevelClient.java:1741)
at com.esdemo.config.ElasticSearchClientConfig.restHighLevelClient(ElasticSearchClientConfig.java:19)
at com.esdemo.config.ElasticSearchClientConfig$$EnhancerBySpringCGLIB$$cca08d.CGLIB$restHighLevelClient$0(<generated>)
at com.esdemo.config.ElasticSearchClientConfig$$EnhancerBySpringCGLIB$$cca08d$$FastClassBySpringCGLIB$$14e70886.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361)
at com.esdemo.config.ElasticSearchClientConfig$$EnhancerBySpringCGLIB$$cca08d.restHighLevelClient(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 19 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.xcontent.DeprecationHandler
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 43 common frames omitted
Process finished with exit code 1
Solution:
Check whether the jar package of ES is introduced into Maven’s POM file
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.7.0</version>
</dependency>If there is still an error after introducing the jar package, add the following to the startup class
System.setProperty("es.set.netty.runtime.available.processors", "false");
Solution:
https://github.com/goharbor/harbor/issues/12003
postgresql.conf:
sudo sed 's/max_connections =.*/max_connections=999/g'
This issue is because the postgres database connection exceeded.
The default max connection is 100.
However ever it's still not working even set database.max_open_conns in harbor.yaml.
You have to also manually edit postgresql.conf like
sudo sed 's/max_connections =.*/max_connections=999/g' /data/database/postgresql.conf
and restart harbor-db
track db issue with #12124
Lombok is used, and the compilation is normal, but the idea cannot be started, saying that the builder method does not exist. At this time, just set the idea:
-Djps.track.ap.dependencies=false
Es reports an error when executing aggregate query: alternately, set fielddata = true on
GET /megacorp/employee/_search
{
"aggs": {
"all_interests": {
"terms": { "field": "interests" }
}
}
}
Execute the following command to start fielddata
PUT /your_index_name/_mapping?pretty
{
"properties": {
"your_field_name": {
"type": "text",
"fielddata": true
}
}
}
./node_modules/antd/es/style/index.less (./node_modules/css-loader/dist/cjs.js??ref–6-oneOf-7-1!./node_modules/postcss-loader/src??postcss!./node_modules/resolve-url-loader??ref–6-oneOf-
7-3!./node_modules/less-loader/dist/cjs.js??ref–6-oneOf-7-4!./node_modules/antd/es/style/index.less)
ValidationError: Invalid options object. Less Loader has been initialized using an options object that does not match the API schema.
This is the problem with the less-loader version
yarn remove less-loader
yarn add [email protected]This problem occurs because after elasticsearch configures the password, the connection es encounters authentication problems when logstash starts. The solution requires configuring the account password in the logstash configuration file
vim /app/logstash/config/beat_es.conf
input {
beats {
port => 5044
}
}
filter {
#Only do json parsing on nginx json logs, system messages are in other formats, no need to handle
if [fields][log_type] == "nginx"{
json {
source => "message"
remove_field => ["beat","offset","tags","prospector"] #Remove fields, no collection required
}
date {
match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] # match the timestamp field
target => "@timestamp" # write the matched data to the @timestamp field
}
}
}
output {
if [fields][log_type] == "ruoyi" {
elasticsearch {
hosts => ["node1:9200","node2:9200"]
user => elastic
password => "123123"
index => "ruoyi_log"
timeout => 300
}
}
}