24322; normal information
Found interface org.elasticsearch.common.bytes.BytesReference, but class was expected
picture of child problems.
To solve the solution,
<elasticsearch.version> 7.6.2
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
When using elasticsearch word segmentation plug-in, kibana is used as the client management, and an error kibana did not load properly. Check the server output for more information
Failed to load resource: Net:: err_ CACHE_ READ_ FAILURE。 Cache read failed.
The reason is that I cleared the browser’s cache before and restarted the computer, which resulted in kibana’s failure to read the cache.
Solution: force the browser cache to refresh again.
Exception in thread "main" ElasticsearchStatusException[Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.]];
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1727)
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1704)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1467)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1424)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1394)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:930)
at com.softsec.util.demoTime.main(demoTime.java:98)
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://192.168.101.92:9200], URI [/news/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"news","node":"8GuMfo5aRz2CCgl49bY0aQ","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [content_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:253)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:231)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:205)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1454)
... 4 moreThis is because the string (content_type) type of my grouping aggregate query is of type Text
Reason analysis:
When using term query, the type of the query keyword on ES must be keyword rather than text because it is an accurate match. For example, if your search condition is “name” : “Cai Xukun”, then the ES type of the name field must be keyword rather than text
In es, only the keyword type string can use AggregationBuilders. Terms (” aggs – class “) to group aggregation and grouping want to query, according to the specified keyword attribute of grouping field is ok (below);
How do we change that in our Java code?Below, just add “.keyword”
Add to the previous error:
Nginx-style IK hot deployment and its messy code problems
1. Create dic file
2 under the sibling directory of nginx. Adds a remote participle to the ik participle configuration file
After 
is configured, restart nginx and elastic search.
Linux system configuration is the same as window configuration
if there is a problem with messy code, please see the following messy code solution
Messy code problem solved, pro – test available
1. Add nginx to Server
default_type 'text/html'; charset utf-8;
Add the following code to Elastic
-Xms1g
-Xmx1g
-Dfile.encoding=GBK
Processes configured in supervisor cannot start properly
supervisorctl status cerebro FATAL Exited too quickly (process log may have details)
There is so little information here that we need to go to the specific log to see what went wrong.
tail -20 /var/log/supervisord.log
2017-08-07 13:23:36,829 INFO spawned: 'cerebro' with pid 16482
2017-08-07 13:23:36,863 INFO exited: cerebro (exit status 1; not expected)
2017-08-07 13:23:36,863 INFO gave up: cerebro entered FATAL state, too many start retries too quicklyThere is more information here, but there is nothing of substance.
So what happens when you boot it up
cupname stdout
> this command is the dynamic output when the process is launched
/usr/bin/env: bash: Not a directoryEnvironment variables are not configured correctly.
I have already configured the Java environment variable in /etc/profile, but it has no effect.
The supervisor does not load /etc/profile when it starts. (The design itself)
But supervisor provides a configuration parameter, Enviroment
[program:cerebro]
environment = JAVA_HOME="/opt/jdk/"
command =/bin/bash /opt/cerebro/bin/cerebro
autostart =true
autorestart = true
use
supervisorctl update
supervisorctl reload
supervisorctl statusProcess working properly
cerebro RUNNING pid 17236, uptime 0:00:08
elasticsearch RUNNING pid 17235, uptime 0:00:08
1 Usage Scenario
In some projects, the index of ES needs to be dynamically generated according to the conditions in the program to achieve the purpose of data collation. For example, a system log with a large amount of data needs to be indexed, so index needs to be generated dynamically.
2 Spel generates Index dynamically
Here USES the spring - data - elasticsearch </ code> ElasticsearchRestTemplate </ code> es operation, version for 3.2.0.
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>3.2.0.RELEASE</version>
</dependency>
You can specify the static IndexName when identifying POJO objects using the @document annotation.
@Document(indexName = "indexName", type = "logger")
But how do you dynamically generate an index?So let’s say Spel.
Spel full name is Spring Expression Language, is a Spring Expression Language, function is very powerful, official website.
Using Spel, you can call the Bean’s methods through an expression in the annotation to assign values to parameters.
Therefore, the idea of dynamic generation is to create an index generator and call the generator method in @ducument to assign the attribute indexName.
Examples of generators are as follows:
@Component("indexNameGenerator")
public class IndexNameGenerator {
public String commonIndex() {
String date = LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd"));
return "collectlog_" + date;
}
}
An instance of the Document class is as follows:
@NoArgsConstructor
@Data
@Document(indexName = "#{@indexNameGenerator.commonIndex()}", type = "logger")
public class ESDocument {
@Field(analyzer = "ik_max_word", searchAnalyzer = "ik_max_word")
private String content;
private String cluster;
private long date = LocalDateTime.now().toInstant(ZoneOffset.ofHours(8)).toEpochMilli();
private String path;
private String ip;
private String level;
}
As you can see, in the @ the Document </ code> annotation, call the indexNameGenerator.com monIndex () </ code>, the method to obtain the Index of every day.
this method is called every time new data is added.
3. Some notes
Note that Spel USES context to get the corresponding Bean Resolver. If the project is running and an exception is reported,
org.springframework.expression.spel.SpelEvaluationException:
EL1057E: No bean resolver registered in the context to resolve access to bean 'indexNameGenerator'
This reason is mainly caused by problems with the spring context used in Spel, you can break to the SpelExpression#getValue method debugging problem.
org.springframework.expression.spel.standard.SpelExpression#getValue(org.springframework.expression.EvaluationContext, java.lang.Class<T>)
When used in ES, if is to manually create the ElasticsearchRestTemplate </ code>, must be set when creating the instance of ElasticsearchConverter </ code>, or you will quote the exception.
ElasticsearchConverter can use the generated Bean in springboot autoconfig, in which the correct ApplicationContext has been injected. The example is as follows:
@EnableConfigurationProperties(ESProperties.class)
@Service("elasticSearchHandlerFactory")
public class ElasticSearchHandlerFactory {
@Resource
ESProperties esProperties;
@Resource
ElasticsearchConverter elasticsearchConverter;
public ElasticsearchRestTemplate getHandler() {
ESProperties.DbConfig dbConfig = esProperties.getDbList().get("c1");
final ClientConfiguration configuration = ClientConfiguration.builder()
.connectedTo(dbConfig.getHostAndPort())
.withBasicAuth(dbConfig.getUsername(), dbConfig.getPassword())
.build();
RestHighLevelClient client = RestClients.create(configuration).rest();
return new ElasticsearchRestTemplate(client, elasticsearchConverter);
}
}
The above content is a personal learning summary, if there is any improper, welcome to correct in the comments
GET /oms_historyalarm.historyalarm_recent/historyalarm_recent/_search
{
"query":{
"range":{
"fault_start_time":{
"from":"2019-10-28 00:00:00",
"to":"2019-10-28 17:46:03",
"format": "yyyy-MM-dd hh:mm:ss"
}
}
}
}
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
GET /oms_historyalarm.historyalarm_recent/historyalarm_recent/_search
{
"query":{
"range":{
"fault_start_time.keyword":{
"from":"2019-10-28 00:00:00",
"to":"2019-10-28 17:46:03",
"format": "yyyy-MM-dd hh:mm:ss"
}
}
}
}
[2018-11-20T06:38:34,516][ERROR][o.e.b.Bootstrap ] [es-node] Exception
org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:771) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:736) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:173) ~[?:?]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:209) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.node.Node.start(Node.java:694) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:278) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.6.2.jar:5.6.2]
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:?]
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980) ~[?:?]
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250) ~[?:?]
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365) ~[?:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
[2018-11-20T06:38:34,546][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es-node] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to bind to [9300-9400]]; nested: BindException[Cannot assign requested address];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.2.jar:5.6.2]
Caused by: org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:771) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:736) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:173) ~[?:?]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:209) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.node.Node.start(Node.java:694) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:278) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.2.jar:5.6.2]
... 6 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:?]
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:554) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1258) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:980) ~[?:?]
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:250) ~[?:?]
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:365) ~[?:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
[2018-11-20T06:38:35,280][INFO ][o.e.n.Node ] [es-node] stopping ...
[2018-11-20T06:38:35,300][INFO ][o.e.n.Node ] [es-node] stopped
[2018-11-20T06:38:35,301][INFO ][o.e.n.Node ] [es-node] closing ...
[2018-11-20T06:38:35,342][INFO ][o.e.n.Node ] [es-node] closed
Elasticsearch.yml configuration file for
elasticsearch.yml is as follows:
cluster.name: es-cluster
node.name: es-node
path.data: /home/bigdata/cluster/elasticsearch/data
path.logs: /home/bigdata/cluster/elasticsearch/logs
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 192.168.159.7
discovery.zen.ping.unicast.hosts: ["linux"]
There should be a space between the
colon and the attribute value (note when configuring). The host name is Linux. The reason for this problem is that the value of the configuration item of net.host cannot be written as the host name Linux, that is, network. Use IP addresses, not hostnames, when possible during configuration.
as shown in the figure below:
through the web page can access http://192.168.159.7:9200
server power off, es directly hung up, when started again, due to the previous data volume is too large, about 500g, stand-alone version es startup recovery data is very difficult, JVM recovery frequent.
[2019-06-17T16:24:14,303][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][20] overhead, spent [319ms] collecting in the last [1s]
[2019-06-17T16:26:05,781][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][131] overhead, spent [263ms] collecting in the last [1s]
[2019-06-17T16:28:39,668][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][284] overhead, spent [337ms] collecting in the last [1.1s]
[2019-06-17T16:29:38,902][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][343] overhead, spent [272ms] collecting in the last [1s]
[2019-06-17T16:30:31,004][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][395] overhead, spent [305ms] collecting in the last [1s]
[2019-06-17T16:32:14,218][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][498] overhead, spent [359ms] collecting in the last [1s]
[2019-06-17T16:33:16,289][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][560] overhead, spent [421ms] collecting in the last [1s]
[2019-06-17T16:35:25,630][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][689] overhead, spent [286ms] collecting in the last [1s]
[2019-06-17T16:36:08,858][INFO ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][old][727][3] duration [5.7s], collections [1]/[6.1s], total [5.7s]/[6.3s], memory [3.2gb]->[2.2gb]/[7.8gb], all_pools {[young] [839.5mb]->[8.7mb]/[865.3mb]}{[survivor] [108.1mb]->[0b]/[108.1mb]}{[old] [2.2gb]->[2.2gb]/[6.9gb]}
[2019-06-17T16:36:08,859][WARN ][o.e.m.j.JvmGcMonitorService] [node-1] [gc][727] overhead, spent [5.7s] collecting in the last [6.1s]
[2019-06-17T16:36:09,146][WARN ][o.e.i.c.IndicesClusterStateService] [node-1] [[audit-net-udpflow-2019-05-24][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.indices.recovery.RecoveryFailedException: [audit-net-udpflow-2019-05-24][0]: Recovery failed on {node-1}{bxqdAUPETm6og34d591R-Q}{93eABCNYTVioLPSVZlfbYA}{172.16.211.130}{172.16.211.130:9310}{rack=r1}
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2019) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:568) [elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: org.elasticsearch.index.shard.IndexShardRecoveryException: failed to recover from gateway
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:402) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: failed to open reader on writer
at org.elasticsearch.index.engine.InternalEngine.createSearcherManager(InternalEngine.java:564) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:229) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:154) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:2140) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:2122) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:1335) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:1295) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:395) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more
Caused by: java.io.IOException: Map failed: MMapIndexInput(path="/TongYou/hbdata/data/nodes/0/indices/aNIh4sZLQ8WjmVCqeitB8w/0/index/_1in2.fdt") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 753581538 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940) ~[?:?]
at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:267) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:242) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:99) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:150) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:125) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:258) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:490) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.elasticsearch.index.engine.InternalEngine.createSearcherManager(InternalEngine.java:550) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:229) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:154) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:2140) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:2122) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:1335) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:1295) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:395) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more
[2019-06-17T16:36:09,163][WARN ][o.e.c.a.s.ShardStateAction] [node-1] [audit-net-udpflow-2019-05-24][0] received shard failed for shard id [[audit-net-udpflow-2019-05-24][0]], allocation id [MP_rxUh6Slu1ZGwpoGFtog], primary term [0], message [failed recovery], failure [RecoveryFailedException[[audit-net-udpflow-2019-05-24][0]: Recovery failed on {node-1}{bxqdAUPETm6og34d591R-Q}{93eABCNYTVioLPSVZlfbYA}{172.16.211.130}{172.16.211.130:9310}{rack=r1}]; nested: IndexShardRecoveryException[failed to recover from gateway]; nested: EngineCreationFailureException[failed to open reader on writer]; nested: IOException[Map failed: MMapIndexInput(path="/TongYou/hbdata/data/nodes/0/indices/aNIh4sZLQ8WjmVCqeitB8w/0/index/_1in2.fdt") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 753581538 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]]; ]
org.elasticsearch.indices.recovery.RecoveryFailedException: [audit-net-udpflow-2019-05-24][0]: Recovery failed on {node-1}{bxqdAUPETm6og34d591R-Q}{93eABCNYTVioLPSVZlfbYA}{172.16.211.130}{172.16.211.130:9310}{rack=r1}
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2019) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:568) [elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: org.elasticsearch.index.shard.IndexShardRecoveryException: failed to recover from gateway
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:402) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: failed to open reader on writer
at org.elasticsearch.index.engine.InternalEngine.createSearcherManager(InternalEngine.java:564) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:229) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:154) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:2140) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:2122) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:1335) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:1295) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:395) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more
Caused by: java.io.IOException: Map failed: MMapIndexInput(path="/TongYou/hbdata/data/nodes/0/indices/aNIh4sZLQ8WjmVCqeitB8w/0/index/_1in2.fdt") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 753581538 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940) ~[?:?]
at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:267) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:242) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:99) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:150) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:125) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:258) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:490) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42]
at org.elasticsearch.index.engine.InternalEngine.createSearcherManager(InternalEngine.java:550) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:229) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:154) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:2140) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:2122) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:1335) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:1295) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:395) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:94) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:292) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:92) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1577) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2015) ~[elasticsearch-6.1.2-SNAPSHOT.jar:6.1.2-SNAPSHOT]
... 4 more1. Ulimit-v and ulimit-m must return unlimited.
2.vm.max_map_count is a little bigger than the error message above.
view (root user) : sysctl-a |grep vm.max_map_count
temporary setting: sysctl-w vm.max_map_count= XXXXX
permanent setting:
vi/etc/sysctl. Conf
Add a line
file
vm. Max_map_count = 262144
3. JVM tuning.
reference:
Elasticsearch performance tuning
Elasticsearch 6.0 performance tuning strategy
4. If the server memory is not enough, you can only back up the es data, rebuild it, and manually restore it by using the script, which is particularly troublesome.
Elasticsearch eliminates unssigned, solve red problem – pit
view the docker container address
docker inspect --format='{{.Name}} - {{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $(docker ps -aq)