Download kafka
website: http://kafka.apache.org/
enter the url: http://kafka.apache.org/downloads
System environment
1, the operating system: 64 CentOS7
2, JDK version: 1.8.0 comes with _271 version
3, they are: they are – 3.4.6
4, three server deployment success: 192.168.192.150; 192.168.192.152; 192.168.192.155;
Pre-setting:

[root@localhost /]# timedatectl set-timezone Asia/Shanghai
root@localhost /]# timedatectl
[root@localhost /]# timedatectl
[root@localhost /]# timedatectl
[root@localhost /]# timedatectl

[root@master /]# yum-y install NTP
p1
[>daoplaceholder1 /]# ntpdate ntp1.aliyun.com
hostname ectl set-hostname master
stname ectl set-hostname node1
h>ame ectl set-hostname node2
hostname ectl set-hostname node2
hostname ectl set-hostname node2
hostname ectl set-hostname node2

192.168.192.150 master
2.168.192.152 node1
1>68.192.155 node2
192.168.192.155

127.0.0.1 localhost
: 1 localhost
1>68.93.150 master
192.168>152 node1
192.168.93.15>de2
192.168.93.155 node2
192.168.93.155 node2

192.168.93.155 nod> br> If you change the hosts in Windows 10, you can copy this file to the desktop and add the above contents. Then replace the hosts file under C:\Windows\System32\ Drivers \etc with
.

props. Put (consumerConfig. Bootstrap_servers_config,”node1:9092,node2:9092″);
tar-zxvf /usr/local/soft/kafka_2.11-2.2.0.tgz -c /usr/local/ kafka
tar-zxvf /usr/local/soft/kafka_2.11-2.2.0.tgz -c /usr/local/
2, configuration, kafka
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Kafka’s configuration information is configured in server.properties
Find the following two lines of code and comment them separately
#broker.id=0
#zookeeper.connect=localhost:2181
Modify the directory where the logs are stored
The log. The dirs =/kafka_2. 11-2.2.0/kafka – logs
Add the following three configurations at the bottom of the file:
Broker. Id = 1
zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181

Listeners = PlainText :// Master :9092 Listeners :// Master :9092

[root@localhost local]# CD kafka2.11-2.2.0 /
root@localhost kafka2.11-2.2.0]# mkdir kafka-logs
[root@localhost local]# CD kafka2.11-2.2.0 /
[root@localhost kafka2.11-2.2.0]# mkdir kafka-logs

Note: If it is a stand-alone version, the default is fine, we do not need to change anything. Now we are configuring the cluster, so we need to configure some parameters
1), Broker. id: Each machine cannot be the same
2), ZooKeeper. Connect: Since I have 3 ZooKeeper servers, I set ZooKeeper. Connect to 3 servers and must add all of them
3), Listeners: Listeners must be set when configuring the cluster, otherwise the leader error will not be found for future operations
WARN [Producer clientId = console – Producer] Error while fetching the metadata with the correlation id 40: {test = LEADER_NOT_AVAILABLE} workClient (org.apache.kafka.clients.Net)
4) Notice that the two servers, ZooKeeper. Connect, are configured the same as here, but the Broker. ID and Listeners cannot be configured the same
5 copies, kafka to two other servers
[root @ localhost config] # SCP – r/usr/local/kafka_2. 11-2.2.0 [email protected]:/usr/local/
[root @ localhost config] # SCP – r/usr/local/kafka_2. 11-2.2.0 [email protected]:/usr/local /
You will be asked to enter the password of the target machine. Just follow the instructions and modify the Broker. id and Listeners of the two servers as follows:
6, 192.168.192.152
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Then a change on the broker. Id = 2 and listeners = PLAINTEXT:// 192.168.192.152:9092, the zookeeper. Connect don’t need to change are the same
Broker.
id = 2 zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181
listeners = PLAINTEXT:// node1:9092
7, 192.168.192.155
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Then a change on the broker. Id = 3 and listeners = PLAINTEXT:// 192.168.192.155:9092, the zookeeper. Connect don’t need to change are the same
Broker. Id = 3
zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181
listeners = PLAINTEXT:// 2:9092
If the firewall has been turned off, it can be ignored.
All three machines must be turned on. Kafka communication is conducted through port 9092 by default, which is the Listed Listeners we have provided above
[root@localhost config]# firewall-cmd –zone=public –add-port=9092/tcp –permanent
[root@localhost config]# firewall-cmd –reload

[root@localhost /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
[root@localhost /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
[root@localhost /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
10, start, kafka,
three is to launch the
[root @ localhost /] #/usr/local/kafka_2. 11-2.2.0/bin/kafka – server – start. Sh – daemon/usr/local/kafka_2. 11-2.2.0/config/server properties

[root@localhost /]# JPS
87 JPS
9>kafka
9224 Kafka
9224 Kafka
[root@localhost /]# CD /usr/local/kafka_2.11-2.2.0/
root@localhost kafka_2.11-2.2.0]# bin/ kafka-Topics. Sh –create — ZooKeeper 192.168.192.150:2181 –replication-factor 1 –partitions 1 –topic test
Created topic test.
If successful, it outputs: Created Topic “test”.
13, see the topic
while 192.168.192.150 created on the topic, but the other two machine also can see that the 192.168.192.152 client
[root@localhost /]# CD /usr/local/kafka_2.11-2.2.0/
[root@localhost kafka_2.11-2.2.0]# bin/ kafka-Topics — List — ZooKeeper 192.168.192.152:2181
st
Note: The IP here can be 192.168.192.150, 192.168.192.152, 192.168.192.155. Themes can be seen on any of the three servers


root@localhost kafka_2.11-2.2.0]# bin/kafka-console-producer.sh –broker — list 192.168.192.150:9092 –topic test
>> a
> b
> c
# bin/kafka-console-consumer.sh –bootstrap-server 192.168.192.152:9092 — Topic test –from — Beginning
a

c>