Installation Kafka-2.13 CentOS 7.9
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">一、Installation Kafka-2.13 CentOS 7.9</span></h3><p><span style="font-family: "Microsoft YaHei"; font-size: 16px">地址</span></p>
<ul>
<li><span style="font-family: "Microsoft YaHei"; font-size: 16px">https://kafka.apache.org/downloads.html</span></li>
</ul>
<p><img src="https://img2022.cnblogs.com/blog/2412541/202210/2412541-20221012225756229-996819823.png" alt="" height="392" width="520"></p>
<p id="1665586683228"></p>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">二、安装准备</span></h3>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>1</strong> 安装JDK</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">在安装<span style="color: rgba(228, 55, 233, 1)">kafka</span>之前必须先安装JDK和zookeeper，如何安装JDK，可以查看：CentOS 7.9 安装 jdk-8u333</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>2 </strong>下载安装zookeeper</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">如何在CentOS 7 下安装zookeeper，可以查看：CentOS 7.9 安装 zookeeper-3.6.3</span></p>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">二、kafka</span></h3>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>1</strong> 进入Apache官网 <code>http://kafka.apache.org/downloads.html</code> 选择Binary downloads，选择版本进行下载。</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>2</strong> wget下载</span></p>
<pre class="language-bash highlighter-hljs"><code>wget https://archive.apache.org/dist/kafka/3.0.1/kafka_2.13-3.0.1.tgz</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>3</strong> 解压</span></p>
<pre class="language-bash highlighter-hljs"><code>tar -zxvf /opt/software/kafka_2.13-3.0.1.tgz -C /opt/
ll /opt/kafka_2.13-3.0.1/</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>4</strong> 进入kafka目录</span></p>
<p><img src="https://img2022.cnblogs.com/blog/2412541/202210/2412541-20221012232601229-306426347.png" alt="" height="220" width="500"></p>
<p id="1665588367368"></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>5 </strong>启动kafka之前要确保<span style="color: rgba(228, 55, 233, 1)">zookeeper</span>已经启动，如果没有启动，执行以下命令</span></p>
<pre class="language-bash highlighter-hljs"><code>zkServer.sh start</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>6</strong> 修改kafka配置文件中的zookeeper地址，打开配置文件</span></p>
<pre class="language-bash highlighter-hljs"><code>vim /opt/kafka_2.13-3.0.1/config/server.properties
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
############################# Socket Server Settings #############################
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = listener_name://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://192.168.0...:8091
# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured.Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
#advertised.listeners=PLAINTEXT://your.host.name:9092
# A comma separated list of directories under which to store log files
log.dirs=/opt/kafka_2.13-3.0.1/logs
############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=192.168.0...:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=18000</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>7</strong> 新建日志储存路径</span></p>
<pre class="language-bash highlighter-hljs"><code>mkdir /opt/kafka_2.13-3.0.1/logs</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>8</strong> 启动kafka</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-server-start.sh config/server.properties
# 后台启动
./bin/kafka-server-start.sh -daemon config/server.properties</code></pre>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">四、设置开机自动启动</span></h3>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>1</strong> 切换到/lib/systemd/system/目录，创建自启动文件</span></p>
<pre class="language-bash highlighter-hljs"><code>vim /lib/systemd/system/kafka.service
Description=kafkaservice
After=network.target
WorkingDirectory=/opt/kafka_2.13-3.0.1
ExecStart=/opt/kafka_2.13-3.0.1/bin/kafka-server-start.sh /opt/kafka_2.13-3.0.1/config/server.properties
ExecStop=/opt/kafka_2.13-3.0.1/bin/kafka-server-stop.sh
User=root
Group=root
Restart=always
RestartSec=10
WantedBy=multi-user.target</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>2</strong> 设置自启动</span></p>
<pre class="language-bash highlighter-hljs"><code>systemctl enable kafka.service</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>3</strong> 立即启动服务</span></p>
<pre class="language-bash highlighter-hljs"><code>systemctl start kafka.service</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>4</strong> 查看启动状态</span></p>
<pre class="language-bash highlighter-hljs"><code>systemctl status kafka.service</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>5 </strong>关闭服务</span></p>
<pre class="language-bash highlighter-hljs"><code>systemctl stop kafka.service</code></pre>
<p> </p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 18px"><strong>五、Kafka命令测试</strong></span></p>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>1</strong> 创建topic</span></p>
<ul>
<li><span style="font-family: Microsoft YaHei"><span style="font-size: 16px">create：创建topic</span></span></li>
<li><span style="font-size: 16px; font-family: "Microsoft YaHei"">partitions：topic创建时，若不指定<span style="color: rgba(228, 55, 233, 1)">分区个数</span>，则使用server.properties中配置的num.partitions值，也可以自己指定</span></li>
<li><span style="font-size: 16px; font-family: "Microsoft YaHei"">replication：用来设置主题的<span style="color: rgba(228, 55, 233, 1)">副本数</span>。每个主题可以有多个副本，副本位于集群中不同的broker上，也就是说副本的数量不能超过<span style="color: rgba(228, 55, 233, 1)">broker</span>的数量，否则创建主题时会失败。</span></li>
</ul>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --bootstrap-server 192.168.0...:8091 --create --topic iyuyixyz --partitions 2 --replication-factor 1</code></pre>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>2</strong> 列出所有topic</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --bootstrap-server 192.168.0...:8091 --list</code></pre>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>3</strong> 列出所有topic的信息</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --bootstrap-server 192.168.0...:8091 --describe</code></pre>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>4</strong> 列出指定topic的信息</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --bootstrap-server 192.168.0...:8091 --describe --topic iyuyixyz</code></pre>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>5</strong> 生产者（消息发送程序）</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-console-producer.sh --broker-list 192.168.0...:8091 --topic iyuyixyz</code></pre>
<p><span style="font-size: 16px; font-family: "Microsoft YaHei""><strong>6</strong> 消费者（消息接收程序）</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-console-consumer.sh --bootstrap-server 192.168.0...:8091 --topic iyuyixyz</code></pre>
<p> </p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 18px"><strong>六、配置系统环境变量</strong></span></p>
<pre class="language-bash highlighter-hljs"><code>vim /etc/profile
export PATH=$PATH:/opt/kafka_2.13-3.0.1/bin
# 使配置生效
source /etc/profile</code></pre>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">X、One Step Success</span></h3>
<p> </p>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">Y、Error message</span></h3>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>1</strong> zookeeper is not a recognized option</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">新版已不支持zookeeper参数，需要换成bootstrap-server参数。</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --create --zookeeper 192.168.0.98:2181 --replication-factor 1 --partitions 1 --topic iyuyixyz</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">原来新版本的<span style="color: rgba(228, 55, 233, 1)">kafka</span>，已经不需要依赖zookeeper来创建topic，新版的kafka创建topic指令为下</span></p>
<pre class="language-bash highlighter-hljs"><code>./bin/kafka-topics.sh --bootstrap-server 192.168.0...:8091 --create --topic iyuyixyz --partitions 2 --replication-factor 1</code></pre>
<p><img src="https://img2022.cnblogs.com/blog/2412541/202210/2412541-20221013013640576-937994747.png" alt="" height="275" width="600"></p>
<p id="1665596206115"></p>
<p> </p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px"><strong>2</strong> 节点响应超时(请求超时)</span></p>
<pre class="language-bash highlighter-hljs"><code> WARN Connection to node -1 (/192.168.0...:8092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
WARN Connection to node -1 (/192.168.0...:8092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
WARN Connection to node -1 (/192.168.0...:8092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
Error while executing topic command : Timed out waiting for a node assignment. Call: createTopics
ERROR org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics
(kafka.admin.TopicCommand$)</code></pre>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">该错误是由于没有用对应版本的命令，访问了错的主机名（或IP）和端口，所以请求失败导致超时。</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">其中的<span style="color: rgba(228, 55, 233, 1)">createTopics</span>、<span style="color: rgba(228, 55, 233, 1)">listTopics</span>等都会出现该错误。如果没配置server.properties文件的<span style="color: rgba(228, 55, 233, 1)">listeners</span>值，也会报以上错误。</span></p>
<p><span style="font-family: "Microsoft YaHei"; font-size: 16px">listeners值默认是PLAINTEXT://:9092，要改为PLAINTEXT://localhost:9092或<span style="color: rgba(228, 55, 233, 1)">PLAINTEXT://IP:9092</span>等</span></p>
<p><img src="https://img2022.cnblogs.com/blog/2412541/202210/2412541-20221013012009167-2069021145.png" alt="" height="219" width="600"></p>
<p id="1665592503762"></p>
<p id="1665592497003"></p>
<p> </p>
<h3><span style="font-family: "Microsoft YaHei"; font-size: 18px">Z、Related Links</span></h3>
<p> </p><br><br>
来源：https://www.cnblogs.com/huaxiayuyi/p/16786470.html
頁:
[1]