After the kafka consumer starts, I get the exception as the title.
Looking up the data, I found out that
__consumer_offsetsthis topic is abnormal. The
abnormality may be that the topic has been deleted and no longer exists, or it may be that the topic is abnormal due to partition or other reasons.
Go to zookeeper to see if this topic exists.
ls /brokers/topics [__consumer_offsets , xx,xx]
1) __consumer_offsets does not exist
./kafka-topics.sh --zookeeper master:2181 --partitions 1 --replication-factor 1 --create --topic __consumer_offsets
If topic does not exist, create one directly.
2) __consumer_offsets exists
Mine is this problem, topic exists, but an exception occurred during kafka migration.
- Then you need to stop kafka first.
- delete topic in zookeeper
# remove node infos deleteall /brokers/topics/__consumer_offsets # remove node delete /brokers/topics/__consumer_offsets
- Restart kafka.
In theory, restart kafka, the consumer will go online automatically, and a
__consumer_offsetstopic will be created automatically. If not, follow the previous step and create one manually.