Error when starting kafka
The problem occurs in the logs file
ERROR Shutdown broker because all log dirs failed: It means that all log files cannot be used, which may be a permission problem
Then enter the logs to view
Currently our user is hadoop, and the log files belong to the root user, so it can be solved by modifying the permissions
sudo chown hadoop.hadoop logs -R
You can see that the file ownership has been changed to hadoop
try to start kafka
successfully started
ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.InconsistentBrokerIdException: Configured broker.id 0 doesn’t match stored broker.id Some(1) in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs). at kafka.server.KafkaServer.getOrGenerateBrokerId(KafkaServer.scala:793) at kafka.server.KafkaServer.startup(KafkaServer.scala:221) at kafka.Kafka$.main(Kafka.scala:109) at kafka.Kafka.main(Kafka.scala)
2. Casue
The id value inside meta.properties (path: /opt/kafka/logs) does not match the broker.id in server.properties of /opt/kafka/config.
The cause is this: due to the mistaken deletion of the file on linux, so that _cd_ such a command can not be used, but fortunately there are other nodes can be used, after some backtracking, successfully run zookeeper, however, when running kafka, reported an error ERROR Fatal error during KafkaServer Prepare to shutdown, the first line of the error message is as follows.
The reason for this is that the id value in the meta.properties (path: /opt/kafka/logs) does not match the broker.id in the server.properties in /opt/kafka/config.
This is after I modified it, originally it was broker.id=1
3. System prompt
4. Solution
Found the reason, broker=0 and broker.id=1 modified to the same value, and then restart, ERROR Fatal error during KafkaServer startup.
0. Installation reference
Note that the new version of Kafka does not need to install zookeeper separately. You can directly use the self-contained one. Of course, you can also install it separately
1. Error message
Kafka startup error: java.net.ConnectException: Connection refused
2. Cause of error
Zookeeper was not started before Kafka was started
3. Solution
Start zookeeper first and then Kafka again
4. Start command
4.1 to Kafka installation directory
cd /usr/src/kafka_2.12-3.1.0/
4.2 start zookeeper (the latest version of the zookeeper from Kafka is used here)
bin/zookeeper-server-start.sh config/zookeeper.properties
4.3 start Kafka
bin/kafka-server-start.sh config/server.properties
4.4 close Kafka
bin/kafka-server-stop.sh config/server.properties
Kafka startup error
Solution:
server.properties file
listeners=PLAINTEXT://:9092(Note that there is no IP address in this line, I just had a problem with the IP) advertised.listeners=PLAINTEXT://114.115.20.100:9092 (need to let go of the comment)