Question

After the kafka consumer starts, I get the exception as the title.

Looking up the data, I found out that __consumer_offsetsthis topic is abnormal. The
abnormality may be that the topic has been deleted and no longer exists, or it may be that the topic is abnormal due to partition or other reasons.

Go to zookeeper to see if this topic exists.

ls /brokers/topics
[__consumer_offsets , xx,xx]

1) __consumer_offsets does not exist

./kafka-topics.sh --zookeeper master:2181 --partitions 1 --replication-factor 1 --create --topic __consumer_offsets

If topic does not exist, create one directly.

2) __consumer_offsets exists

Mine is this problem, topic exists, but an exception occurred during kafka migration.

1. Then you need to stop kafka first.
2. delete topic in zookeeper

# remove node infos
deleteall /brokers/topics/__consumer_offsets
# remove node
delete /brokers/topics/__consumer_offsets

3. Restart kafka.

In theory, restart kafka, the consumer will go online automatically, and a __consumer_offsetstopic will be created automatically. If not, follow the previous step and create one manually.