银行麒麟系统离线部署clickhouse三节点集群

用户众妙之门 發表於 2023-6-19 18:50:00

银行麒麟系统离线部署clickhouse三节点集群

在国产化环境下部署clickhouse集群，个人建议先查看cpu的内核信息，可能有的人会说，
去银河麒麟的官网查看适配的版本信息，实践证明，官网的未必就是真的正确
好比银河麒麟的官网是这样介绍的
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619170031036-77746810.png" alt="" loading="lazy">
我在安装的过程中，就会报不支持的二进制文件的错误，所以这里不建议安装官网推荐的版本，毕竟是在国产化的环境下，目前国产化的兼容性还是做得不好。
 
下面我们正式开始介绍在国产化环境中应该如何部署clickhouse三节点集群。
1、查看cpu的内核信息 
<div class="cnblogs_code">
<pre>cat /proc/cpuinfo</pre>
</div>
 
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619170533577-819978415.png" alt="" loading="lazy">
 可以看出，cpu是基于arm的，所以我们下载cllckhouse安装包的时候也要基于arm内核的
 
2、clickhouse安装包下载，下载地址可以参考这里：https://packages.clickhouse.com/tgz/stable/
这里是我本地下载的安装包
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619171053877-1771025748.png" alt="" loading="lazy">
 
3、修改linux文件限制（三台服务器都需要修改）
<div class="cnblogs_code">
<pre>vim /etc/security/limits.conf

//添加以下内容
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072</pre>
</div>
 
 
4、关闭防火墙（三台服务器都需要关闭防火墙）
<div class="cnblogs_code">
<pre>systemctl status firewalld.service//查看防火墙状态
systemctl stop firewalld.service//关闭防火墙
systemctl disable firewalld.service//永久关闭防火墙</pre>
</div>
 
 
5、配置免密通讯
每台服务器都进行以下操作：
<div class="cnblogs_code">
<pre>vim /etc/hosts

//添加以下内容
176.28.40.25 bigdata.exchange01
176.28.40.26 bigdata.exchange02
176.28.40.27 bigdata.exchange03</pre>
</div>
 
 
每台服务器都进行以下操作：
<div class="cnblogs_code">
<pre>ssh-keygen
cd .ssh
cat id_rsa.pub >> authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys</pre>
</div>
 
以其中一台服务器为例
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619172123568-2143569541.png" alt="" loading="lazy">
 这个时候每台服务器就会生成自己的公钥，如果需要能相互之间免密访问，就需要活得别人的公钥，我这里用简单粗暴的方法，就是把别的机器的公钥内容都复制到所在服务器的
<pre> authorized_keys 文件里面 好比这样，每台服务器里面的 authorized_keys 文件里面都存放着另外两台机器的公钥，第一台服务器就把第二第三台的公钥复制过来，第二台服务器就把第一第三台台服务器的公钥 复制过来，同样的道理，第三台就把第一第二的公钥复制过来，authorized_keys 文件所在目录/root/.ssh</pre>
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619172629272-200093505.png" alt="" loading="lazy">
 
 
6、安装zookeeper，安装包可以到官网下载，这里我就不做赘述了
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619173142240-284441361.png" alt="" loading="lazy">
 上传安装包，并解压安装包（三台服务器都这样操作）
<div class="cnblogs_code">
<pre>tar -zxf zookeeper-3.4.5.tar.gz</pre>
</div>
在zookeeper的conf目录，拷贝zoo_sample.cfg为zoo.cfg
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619174330918-1610879681.png" alt="" loading="lazy">
 修改配置文件
<div class="cnblogs_code">
<pre>vim zoo.cfg 
tickTime=10000
initLimit=10
syncLimit=5
dataDir=/opt/softwares/zookeeper/data
dataLogDir=/opt/softwares/zookeeper/logs
autopurge.purgeInterval=0
globalOutstandingLimit=200
clientPort=2181
server.1=176.28.40.25:2888:3888
server.2=176.28.40.26:2888:3888
server.3=176.28.40.27:2888:3888</pre>
</div>
接下来在每台服务器的dataDir=/opt/softwares/zookeeper/data 目录下创建文件myid,分别在相应的myid文件里面写入 1 、2、3
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619175045237-1033644611.png" alt="" loading="lazy">
 另外两台服务器就是2  和  3
<h4>启动zookeeper，启动脚本如下（三台服务器都进去启动）</h4>
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619175228806-741494385.png" alt="" loading="lazy">
<div class="cnblogs_code">
<pre>启动:./zkServer.sh start
查看状态: ./zkServer.sh status</pre>
</div>
 
7、安装clickhouse(三台服务器都进行操作)
把安装包上传到三台服务器，并解压
<div class="cnblogs_code">
<pre>tar -zxf 压缩包名称</pre>
</div>
 
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619180117503-539344067.png" alt="" loading="lazy">
 按照以下顺序进行安装（三台服务器都进行这样的安装）
<div class="cnblogs_code">
tar -xzvf clickhouse-common-static-22.3.10.22.tgz （ClickHouse编译的二进制文件。） clickhouse-common-static-22.3.10.22/install/doinst.sh tar -xzvf clickhouse-common-static-dbg-22.3.10.22.tgz （创建clickhouse-server软连接，并安装默认配置服务） clickhouse-common-static-dbg-22.3.10.22/install/doinst.sh tar -xzvf clickhouse-server-22.3.10.22.tgz （创建clickhouse-client客户端工具软连接，并安装客户端配置文件。） clickhouse-server-22.3.10.22/install/doinst.sh tar -xzvf clickhouse-client-22.3.10.22.tgz（带有调试信息的ClickHouse二进制文件。） clickhouse-client-22.3.10.22/install/doinst.sh

</div>
 
 
这里需要注意一点，在进行clickhouse-server-22.3.10.22/install/doinst.sh安装的时候，会提示设置好default用户的密码，这里设置的时候要记好了，只有一次输入的机会，我这里设置的密码是clickhouse
安装日志如下：
<div class="cnblogs_code">
<pre>ClickHouse binary is already located at /usr/bin/clickhouse
Symlink /usr/bin/clickhouse-extract-from-config already exists but it points to /opt/softwares/clickhouse/clickhouse-server-22.3.10.22/install/clickhouse. Will replace the old symlink to /usr/bin/clickhouse.
Creating symlink /usr/bin/clickhouse-extract-from-config to /usr/bin/clickhouse.
Creating clickhouse group if it does not exist.
groupadd -r clickhouse
groupadd：“clickhouse”组已存在
Creating clickhouse user if it does not exist.
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse clickhouse
useradd：用户“clickhouse”已存在
Will set ulimits for clickhouse user in /etc/security/limits.d/clickhouse.conf.
Config file /etc/clickhouse-server/config.xml already exists, will keep it and extract path info from it.
/etc/clickhouse-server/config.xml has /var/lib/clickhouse as data path.
/etc/clickhouse-server/config.xml has /var/log/clickhouse-server/ as log path.
Users config file /etc/clickhouse-server/users.xml already exists, will keep it and extract users info from it.
Log directory /var/log/clickhouse-server/ already exists.
Data directory /var/lib/clickhouse already exists.
Pid directory /var/run/clickhouse-server already exists.
chown -R clickhouse:clickhouse '/var/log/clickhouse-server/'
chown -R clickhouse:clickhouse '/var/run/clickhouse-server'
chownclickhouse:clickhouse '/var/lib/clickhouse'
groupadd -r clickhouse-bridge
groupadd：“clickhouse-bridge”组已存在
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse-bridge clickhouse-bridge
useradd：用户“clickhouse-bridge”已存在
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-odbc-bridge'
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-library-bridge'
Password for default user is already specified. To remind or reset, see /etc/clickhouse-server/users.xml and /etc/clickhouse-server/users.d.
Setting capabilities for clickhouse binary. This is optional.
Cannot set 'net_admin' or 'ipc_lock' or 'sys_nice' capability for clickhouse binary. This is optional. Taskstats accounting will be disabled. To enable taskstats accounting you may add the required capability later manually.
Allow server to accept connections from the network (default is localhost only), : y
The choice is saved in file /etc/clickhouse-server/config.d/listen.xml.
chown -R clickhouse:clickhouse '/etc/clickhouse-server'

ClickHouse has been successfully installed.

Start clickhouse-server with:
sudo clickhouse start

Start clickhouse-client with:
clickhouse-client --password</pre>
</div>
 
如果说大家在安装的时候真的不小心乱按，忘记密码了，我这里教大家一招，到这个目录下把user.xml文件删除掉，再重新安装一次clickhouse-server,这样就重新输入一个新的密码了
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619181448326-1207550278.png" alt="" loading="lazy">
 
<h4>配置metrika.xml文件</h4>
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619181539738-1839894578.png" alt="" loading="lazy">
<div class="cnblogs_code">
<pre>root@bigdata clickhouse-server]# cat metrika.xml
<yandex>
<clickhouse_remote_servers>
 <gmall_cluster> 
 <shard>
 <internal_replication>true</internal_replication>
 
 <replica>
 <host>176.28.40.25</host>
 <port>9000</port>
 </replica>
 </shard>
 <shard>
 <internal_replication>true</internal_replication>
 <replica> 
 <host>176.28.40.26</host>
 <port>9000</port>
 </replica>
 </shard>
 <shard>
 <internal_replication>true</internal_replication>
 <replica> 
 <host>176.28.40.27</host>
 <port>9000</port>
 </replica>
 </shard>
 </gmall_cluster>
</clickhouse_remote_servers>
<zookeeper>
 <node>
 <host>bigdata.exchange01.yandex.ru</host>
 <port>2181</port>
 </node>
 <node>
 <host>bigdata.exchange02.yandex.ru</host>
 <port>2181</port>
 </node>
 <node>
 <host>bigdata.exchange03.yandex.ru</host>
 <port>2181</port>
 </node>
</zookeeper>

<macros>
 <shard>01</shard> 
 <replica>176.28.40.25</replica>
</macros>
</yandex></pre>
</div>
 
同样的方式把 metrika.xml文件同步到另外两台服务器，同时把该标签改成自己的ip地址
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619181841510-434608145.png" alt="" loading="lazy">
 
修改config.xml文件
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619182142461-1765149034.png" alt="" loading="lazy">
<div class="cnblogs_code">
<pre> <listen_host>::</listen_host>
<include_from>/etc/clickhouse-server/metrika.xml</include_from>

<zookeeper>
 <node>
 <host>176.28.40.25</host>
 <port>2181</port>
 </node>
 <node>
 <host>176.28.40.26</host>
 <port>2181</port>
 </node>
 <node>
 <host>176.28.40.27</host>
 <port>2181</port>
 </node>
</zookeeper>

<remote_servers>

 <tj_dev>
 <shard>

 <replica>
 <host>176.28.40.25</host>
 <port>9000</port>
 <user>default</user>
 <password>clickhosue</password>
 </replica>
 </shard>
<shard>
<replica>
 <host>176.28.40.26</host>
 <port>9000</port>
 <user>default</user>
 <password>clickhosue</password>
</replica>
</shard>
<shard>
<replica>
 <host>176.28.40.27</host>
 <port>9000</port>
 <user>default</user>
 <password>clickhosue</password>
</replica>
</shard>
 </tj_dev>
</remote_servers>


<macros>
 <shard>01</shard>
 <replica>176.28.40.25</replica>
</macros></pre>
</div>
修改完后，把该配置文件同步到另外两台服务器，可以用scp命令进行同步， 此时配置基本完成。
 
8、启动clickhouse集群
在每台服务器上面执行启动命令
 
<div class="cnblogs_code">
<pre> sudo clickhouse start</pre>
</div>
 
 
 
通过以下命令查看clickhouse运行状态
<div class="cnblogs_code">
<pre> sudo clickhouse status</pre>
</div>
 
 
登录clichouse，我们这里通过default用户进行登录
<div class="cnblogs_code">
<pre>clickhouse-client-u default --password clickhouse</pre>
</div>
 
 
这里需要对clickhouse进行说明一下，如果需要在三台服务器都有表和库，我们就需要在创建数据库和数据表的时候指定集群
在集群创建数据库的命令
<div class="cnblogs_code">
<pre>CREATE DATABASE IF NOT EXISTS gfyy on clustertj_dev;</pre>
</div>
 
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619184535698-728198523.png" alt="" loading="lazy">
 
在该数据库下面创建表命令：
<div class="cnblogs_code">
<pre>use gfyy;

CREATE TABLE IF NOT EXISTS exp_person_info on clustertj_dev
(
PERSON_NO Int32,
PERSON_NM String,
ID_NUMBER String,
SEX String,
BIRTH_DATE Date,
DRIVER_LCS_NUMBER String,
PASSPORT_NO String,
HVPS_NO String,
VEHICLE_NO String,
PERMIT_NO String,
TEL_CO String,
ORIGIN String,
HOME_ADDRESS String,
ETPS_SCCD String,
RECORD_TIME Date,
RECORD_EXP_TIME Date,
RFID_NO String,
MASTER_CUSCD String,
AREA_CODE String,
DCL_TYPECD String
)ENGINE = MergeTree()
ORDER BY PERSON_NO
PARTITION BY PERSON_NO
PRIMARY KEY PERSON_NO;</pre>
</div>
<img src="https://img2023.cnblogs.com/blog/1023171/202306/1023171-20230619184736371-1420170133.png" alt="" loading="lazy">
 
可用如下命令查询集群信息，查看每个节点是否正常运行
<div class="cnblogs_code">
<pre>select * from system.clusters</pre>
</div>
  
来源：https://www.cnblogs.com/braveym/p/17491726.html

頁: [1]

琼殿技术论坛's Archiver

银行麒麟系统离线部署clickhouse三节点集群