Centos7 installing Kafka

Download kafka
website: http://kafka.apache.org/
enter the url: http://kafka.apache.org/downloads
System environment
1, the operating system: 64 CentOS7
2, JDK version: 1.8.0 comes with _271 version
3, they are: they are – 3.4.6
4, three server deployment success: 192.168.192.150; 192.168.192.152; 192.168.192.155;
Pre-setting:

[[email protected] /]# timedatectl set-timezone Asia/Shanghai
[email protected] /]# timedatectl
[[email protected] /]# timedatectl
[[email protected] /]# timedatectl
[[email protected] /]# timedatectl

[[email protected] /]# yum-y install NTP
p1
[>daoplaceholder1 /]# ntpdate ntp1.aliyun.com
hostname ectl set-hostname master
stname ectl set-hostname node1
h>ame ectl set-hostname node2
hostname ectl set-hostname node2
hostname ectl set-hostname node2
hostname ectl set-hostname node2

192.168.192.150 master
2.168.192.152 node1
1>68.192.155 node2
192.168.192.155

127.0.0.1 localhost
: 1 localhost
1>68.93.150 master
192.168>152 node1
192.168.93.15>de2
192.168.93.155 node2
192.168.93.155 node2

192.168.93.155 nod> br> If you change the hosts in Windows 10, you can copy this file to the desktop and add the above contents. Then replace the hosts file under C:\Windows\System32\ Drivers \etc with
.

props. Put (consumerConfig. Bootstrap_servers_config,”node1:9092,node2:9092″);
tar-zxvf /usr/local/soft/kafka_2.11-2.2.0.tgz -c /usr/local/ kafka
tar-zxvf /usr/local/soft/kafka_2.11-2.2.0.tgz -c /usr/local/
2, configuration, kafka
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Kafka’s configuration information is configured in server.properties
Find the following two lines of code and comment them separately
#broker.id=0
#zookeeper.connect=localhost:2181
Modify the directory where the logs are stored
The log. The dirs =/kafka_2. 11-2.2.0/kafka – logs
Add the following three configurations at the bottom of the file:
Broker. Id = 1
zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181

Listeners = PlainText :// Master :9092 Listeners :// Master :9092

[[email protected] local]# CD kafka2.11-2.2.0 /
[email protected] kafka2.11-2.2.0]# mkdir kafka-logs
[[email protected] local]# CD kafka2.11-2.2.0 /
[[email protected] kafka2.11-2.2.0]# mkdir kafka-logs

Note: If it is a stand-alone version, the default is fine, we do not need to change anything. Now we are configuring the cluster, so we need to configure some parameters
1), Broker. id: Each machine cannot be the same
2), ZooKeeper. Connect: Since I have 3 ZooKeeper servers, I set ZooKeeper. Connect to 3 servers and must add all of them
3), Listeners: Listeners must be set when configuring the cluster, otherwise the leader error will not be found for future operations
WARN [Producer clientId = console – Producer] Error while fetching the metadata with the correlation id 40: {test = LEADER_NOT_AVAILABLE} workClient (org.apache.kafka.clients.Net)
4) Notice that the two servers, ZooKeeper. Connect, are configured the same as here, but the Broker. ID and Listeners cannot be configured the same
5 copies, kafka to two other servers
[root @ localhost config] # SCP – r/usr/local/kafka_2. 11-2.2.0 [email protected]:/usr/local/
[root @ localhost config] # SCP – r/usr/local/kafka_2. 11-2.2.0 [email protected]:/usr/local /
You will be asked to enter the password of the target machine. Just follow the instructions and modify the Broker. id and Listeners of the two servers as follows:
6, 192.168.192.152
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Then a change on the broker. Id = 2 and listeners = PLAINTEXT:// 192.168.192.152:9092, the zookeeper. Connect don’t need to change are the same
Broker.
id = 2 zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181
listeners = PLAINTEXT:// node1:9092
7, 192.168.192.155
[root @ localhost] # vi/usr/local/kafka_2. 11-2.2.0/config/server properties
Then a change on the broker. Id = 3 and listeners = PLAINTEXT:// 192.168.192.155:9092, the zookeeper. Connect don’t need to change are the same
Broker. Id = 3
zookeeper. Connect = 192.168.192.150:2181192168 192.152:2181192168 192.155:2181
listeners = PLAINTEXT:// 2:9092
If the firewall has been turned off, it can be ignored.
All three machines must be turned on. Kafka communication is conducted through port 9092 by default, which is the Listed Listeners we have provided above
[[email protected] config]# firewall-cmd –zone=public –add-port=9092/tcp –permanent
[[email protected] config]# firewall-cmd –reload

[[email protected] /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
[[email protected] /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
[[email protected] /]# /usr/local/ ZooKeeper -3.4.6/bin/ zkserver.sh start
10, start, kafka,
three is to launch the
[root @ localhost /] #/usr/local/kafka_2. 11-2.2.0/bin/kafka – server – start. Sh – daemon/usr/local/kafka_2. 11-2.2.0/config/server properties

[[email protected] /]# JPS
87 JPS
9>kafka
9224 Kafka
9224 Kafka
[[email protected] /]# CD /usr/local/kafka_2.11-2.2.0/
[email protected] kafka_2.11-2.2.0]# bin/ kafka-Topics. Sh –create — ZooKeeper 192.168.192.150:2181 –replication-factor 1 –partitions 1 –topic test
Created topic test.
If successful, it outputs: Created Topic “test”.
13, see the topic
while 192.168.192.150 created on the topic, but the other two machine also can see that the 192.168.192.152 client
[[email protected] /]# CD /usr/local/kafka_2.11-2.2.0/
[[email protected] kafka_2.11-2.2.0]# bin/ kafka-Topics — List — ZooKeeper 192.168.192.152:2181
st
Note: The IP here can be 192.168.192.150, 192.168.192.152, 192.168.192.155. Themes can be seen on any of the three servers


[email protected] kafka_2.11-2.2.0]# bin/kafka-console-producer.sh –broker — list 192.168.192.150:9092 –topic test
>> a
> b
> c
# bin/kafka-console-consumer.sh –bootstrap-server 192.168.192.152:9092 — Topic test –from — Beginning
a

c>


Read More: