How to Solve canal & MYSQL or “Kafka cannot consume data” Error

Error 1: interaction between canal and MySQL

Explanation: the essential reason is that the same IP generates too many interrupted database connections (exceeding the maximum value of max_connect_errors) in a short time

If the MySQL server continuously receives requests from the same host, and all these continuous requests are unsuccessful, the established connection will be interrupted. When the cumulative value of these continuous requests is greater than When you set the value of max_connect_errors, the MySQL server will block all subsequent requests from this host.

Solution:   Mysqladmin flush hosts – H 127.0.0.1 – uroot – P password

Error 2: Kafka cannot consume data

Reason: the number of partitions of the theme I created is insufficient. You can manually add the same number of partitions as those set in instance.properties in conf/example of canal

Execute the following command:

//1. View the subject details and the number of partitions

kafka-topics.sh –bootstrap-server hadoop102:9092 –describe –topic ODS_BASE_DB_C

//2. Manually add the number of partitions

kafka-topics.sh –bootstrap-server hadoop102:9092 –alter –partitions 4 –topic ODS_BASE_DB_C

Read More: