Centos 7.4下安装FileBeat

吃瓜群众谨慎发言 發表於 2019-10-7 12:21:00

一. 直接yum方式安装
<div class="cnblogs_code"><img id="code_img_closed_309c963e-25ac-458c-b9a3-f820f9b49a7a" class="code_img_closed lazyload" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_309c963e-25ac-458c-b9a3-f820f9b49a7a" class="code_img_opened lazyload" style="display: none" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_309c963e-25ac-458c-b9a3-f820f9b49a7a" class="cnblogs_code_hide">
<pre> 1 # yum install filebeat
 2 Loaded plugins: fastestmirror, langpacks
 3 base | 3.6 kB00:00:00
 4 docker-ce-stable | 3.5 kB00:00:00
 5 elastic-7.x | 1.3 kB00:00:00
 6 extras | 3.4 kB00:00:00
 7 rsyslog_v8 | 2.5 kB00:00:00
 8 updates | 3.4 kB00:00:00
 9 Loading mirror speeds from cached hostfile
10* base: ap.stykers.moe
11* extras: centos.ustc.edu.cn
12* updates: ap.stykers.moe
13 Resolving Dependencies
14 --> Running transaction check
15 ---> Package filebeat.x86_64 0:7.4.0-1 will be installed
16 --> Finished Dependency Resolution
17
18 Dependencies Resolved
19
20 ========================================================================================================================================================================
21 Package Arch Version Repository Size
22 ========================================================================================================================================================================
23 Installing:
24filebeat x86_64 7.4.0-1 elastic-7.x 23 M
25
26 Transaction Summary
27 ========================================================================================================================================================================
28 Install1 Package
29
30 Total download size: 23 M
31 Installed size: 74 M
32 Is this ok : y
33 Downloading packages:
34 filebeat-7.4.0-x86_64.rpm |23 MB00:00:07
35 Running transaction check
36 Running transaction test
37 Transaction test succeeded
38 Running transaction
39 Installing : filebeat-7.4.0-1.x86_64 1/1
40 Verifying: filebeat-7.4.0-1.x86_64 1/1
41
42 Installed:
43 filebeat.x86_64 0:7.4.0-1
44
45 Complete!</pre>
</div>
View Code</div>
 
二. 进入/etc/filebeat下，修改配置文件filebeat.yml，结果如下：
<div class="cnblogs_code">
<pre> 1 # egrep -v "#|^$" /etc/filebeat/filebeat.yml
 2 filebeat.inputs:
 3 - type: log
 4 enabled: true
 5 paths:
 6 - /data/log/127.0.0.1/*.log ##此处是rsyslog日志服务器集中采集本地日志文件
 7 filebeat.config.modules:
 8 path: ${path.config}/modules.d/*.yml
 9 reload.enabled: false
10 setup.template.settings:
11 index.number_of_shards: 1
12 setup.kibana:
13 output.elasticsearch:
14 hosts: ["localhost:9200"]</pre>
</div>
 
三. 官网给出的检查配置文件的TIPS
To test your configuration file, change to the directory where the Filebeat binary is installed, and run Filebeat in the foreground with the following options specified: <code class="literal">./filebeat test config -e</code>. Make sure your config files are in the path expected by Filebeat (see Directory layout), or use the <code class="literal">-c</code> flag to specify the path to the config file.
<div class="cnblogs_code"><img id="code_img_closed_c629718d-9804-4ba7-977e-6f0769cfb0c0" class="code_img_closed lazyload" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_c629718d-9804-4ba7-977e-6f0769cfb0c0" class="code_img_opened lazyload" style="display: none" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_c629718d-9804-4ba7-977e-6f0769cfb0c0" class="cnblogs_code_hide">
<pre> 1 # ./filebeat test config -e -c /etc/filebeat/filebeat.yml
 2 2019-10-05T21:04:54.678+0800 INFO instance/beat.go:607 Home path: Config path: Data path: Logs path:
 3 2019-10-05T21:04:54.678+0800 INFO instance/beat.go:615 Beat ID: 4f580ab1-f94b-44aa-99f4-364e6cf1d3b0
 4 2019-10-05T21:04:54.679+0800 INFO instance/beat.go:903 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat/bin", "data": "/usr/share/filebeat/bin/data", "home": "/usr/share/filebeat/bin", "logs": "/usr/share/filebeat/bin/logs"}, "type": "filebeat", "uuid": "4f580ab1-f94b-44aa-99f4-364e6cf1d3b0"}}}
 5 2019-10-05T21:04:54.679+0800 INFO instance/beat.go:912 Build info {"system_info": {"build": {"commit": "f940c36884d3749901a9c99bea5463a6030cdd9c", "libbeat": "7.4.0", "time": "2019-09-27T07:45:44.000Z", "version": "7.4.0"}}}
 6 2019-10-05T21:04:54.679+0800 INFO instance/beat.go:915 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.9"}}}
 7 2019-10-05T21:04:54.681+0800 INFO instance/beat.go:919 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-30T14:57:02+08:00","containerized":false,"name":"LogServer","ip":["127.0.0.1/8","::1/128","172.33.7.51/24","fe80::a65d:781a:3778:5b17/64","172.17.0.1/16"],"kernel_version":"3.10.0-693.el7.x86_64","mac":["28:6e:d4:88:c7:e1","28:6e:d4:88:c7:e2","02:42:79:dc:1e:df"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":4,"patch":1708,"codename":"Core"},"timezone":"CST","timezone_offset_sec":28800,"id":"be94151089484f0d834d4e8a2e118751"}}}
 8 2019-10-05T21:04:54.681+0800 INFO instance/beat.go:948 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/usr/share/filebeat/bin", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 105115, "ppid": 104989, "seccomp": {"mode":"disabled"}, "start_time": "2019-10-05T21:04:53.660+0800"}}}
 9 2019-10-05T21:04:54.681+0800 INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.4.0
10 2019-10-05T21:04:54.681+0800 INFO idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.4.0' as ILM is enabled.
11 2019-10-05T21:04:54.681+0800 INFO elasticsearch/client.go:170 Elasticsearch url: http://localhost:9200
12 2019-10-05T21:04:54.682+0800 INFO pipeline/module.go:97 Beat name: LogServer
13 2019-10-05T21:04:54.682+0800 ERROR fileset/modules.go:125Not loading modules. Module directory not found: /usr/share/filebeat/bin/module
14 Config OK</pre>
</div>
View Code</div>
 
四. 加入自启动
<div class="cnblogs_code">
<pre># /bin/systemctl daemon-reload
# /bin/systemctl enable filebeat.service
Created symlink from /etc/systemd/system/multi-user.target.wants/filebeat.service to /usr/lib/systemd/system/filebeat.service.</pre>
</div>
 
五. 启动服务
<div class="cnblogs_code"><img id="code_img_closed_485e7c1a-f65e-4c32-852b-f198c497d36a" class="code_img_closed lazyload" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_485e7c1a-f65e-4c32-852b-f198c497d36a" class="code_img_opened lazyload" style="display: none" alt="" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_485e7c1a-f65e-4c32-852b-f198c497d36a" class="cnblogs_code_hide">
<pre> 1 # systemctl start filebeat.service
 2 # systemctl status filebeat.service
 3 ? filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
 4 Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
 5 Active: active (running) since Sat 2019-10-05 21:14:38 CST; 4s ago
 6 Docs: https://www.elastic.co/products/beats/filebeat
 7Main PID: 105264 (filebeat)
 8 CGroup: /system.slice/filebeat.service
 9 +-105264 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/f...
10
11 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.943-0400 INFO instance/beat.go:422 filebeat start running.
12 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.943-0400 INFO registrar/registrar.go:145 Loading registrar data from /.../data.json
13 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.943-0400 INFO registrar/registrar.go:152 States Loaded from registrar: 8
14 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.944-0400 INFO crawler/crawler.go:72 Loading Inputs: 1
15 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.944-0400 INFO log/log.go:118 Starting metrics logging every 30s
16 Oct 05 21:14:38 LogServer filebeat[105264]: 2019-10-05T09:14:38.945-0400 INFO log/input.go:152 Configured paths:
17 Oct 05 21:14:38 LogServer filebeat: 2019-10-05T09:14:38.945-0400 INFO input/input.go:114 Starting input of type: log; ID: 1120...9762598069
18 Oct 05 21:14:38 LogServer filebeat: 2019-10-05T09:14:38.945-0400 INFO crawler/crawler.go:106 Loading and starting Inputs compl... inputs: 1
19 Oct 05 21:14:38 LogServer filebeat: 2019-10-05T09:14:38.945-0400 INFO cfgfile/reload.go:171 Config reloader started
20 Oct 05 21:14:38 LogServer filebeat: 2019-10-05T09:14:38.946-0400 INFO cfgfile/reload.go:226 Loading of config files completed.
21 Hint: Some lines were ellipsized, use -l to show in full.</pre>
</div>
View Code</div>
 
六. 进程检查
 
<div class="cnblogs_code">
<pre>1 # ps -aux | grep filebeat
2 root 1093420.00.6 1608036 52800 ? SslOct06 1:17 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
3 root 1171420.00.0 112664 980 pts/0 S+ 11:44 0:00 grep --color=auto filebeat</pre>
</div>
 
七. 检查启动成功后，到es服务器上查看索引，可以看到新增了一个以filebeat-7.4.0开头的索引，这就代表filesbeat和es能够正常通信了
<div class="cnblogs_code">
<pre>1 # curl 'localhost:9200/_cat/indices?v'
2 health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
3 greenopen .kibana_task_manager_1 rIJCut09QuGYfuRbTUr2Wg 1 0 2 0 12.5kb 12.5kb
4 greenopen .apm-agent-configuration TC9wWs0-R9Owna4G1kkuUw 1 0 0 0 283b 283b
5 greenopen .kibana_1 9AfuMpbeS7mrVGti3rc0pw 1 0 2 0 11.2kb 11.2kb
6 yellow open filebeat-7.4.0-2019.10.05-000001 2nhRfhN8RCG4DO6JaXeoaA 1 1 133314 0 33.9mb 33.9mb</pre>
</div>
 
八. 获取指定索引详细信息
<div class="cnblogs_code">
<pre> 1 # curl -XGET 'localhost:9200/filebeat-7.4.0-2019.10.05-000001?pretty'
 2 {
 3 "filebeat-7.4.0-2019.10.05-000001" : {
 4 "aliases" : {
 5 "filebeat-7.4.0" : {
 6 "is_write_index" : true
 7 }
 8 },
 9 "mappings" : {
10 "_meta" : {
11 "beat" : "filebeat",
12 "version" : "7.4.0"
13 },
14 "dynamic_templates" : [
15 {
16 "labels" : {
17 "path_match" : "labels.*",
18 "match_mapping_type" : "string",
19 "mapping" : {
20 "type" : "keyword"
21 }
22 }
23 }......</pre>
</div>
 
 
 
<div class="titlepage">
<div>
<div>
<h3 class="title">Default paths</h3>
</div>
</div>
</div>
Filebeat uses the following default paths unless you explicitly change them.
<h5> </h5>
<div class="informaltable">
<table border="1" cellpadding="4"><colgroup><col class="col_1"><col class="col_2"><col class="col_3"></colgroup>
<thead>
<tr><th align="left" valign="top">Type</th><th align="left" valign="top">Description</th><th align="left" valign="top">Location</th></tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">
home
</td>
<td align="left" valign="top">
Home of the Filebeat installation.
</td>
<td align="left" valign="top">
<code class="literal">/usr/share/filebeat</code>
</td>
</tr>
<tr>
<td align="left" valign="top">
bin
</td>
<td align="left" valign="top">
The location for the binary files.
</td>
<td align="left" valign="top">
<code class="literal">/usr/share/filebeat/bin</code>
</td>
</tr>
<tr>
<td align="left" valign="top">
config
</td>
<td align="left" valign="top">
The location for configuration files.
</td>
<td align="left" valign="top">
<code class="literal">/etc/filebeat</code>
</td>
</tr>
<tr>
<td align="left" valign="top">
data
</td>
<td align="left" valign="top">
The location for persistent data files.
</td>
<td align="left" valign="top">
<code class="literal">/var/lib/filebeat</code>
</td>
</tr>
<tr>
<td align="left" valign="top">
logs
</td>
<td align="left" valign="top">
The location for the logs created by Filebeat.
</td>
<td align="left" valign="top">
<code class="literal">/var/log/filebeat</code>
</td>
</tr>
</tbody>
</table>
</div>
For the deb and rpm distributions, these paths are set in the init script or in the systemd unit file. Make sure that you start the Filebeat service by using the preferred operating system method (init scripts or <code class="literal">systemctl</code>). Otherwise the paths might be set incorrectly.
  
来源：https://www.cnblogs.com/wintersweet321/p/11625855.html

頁: [1]

琼殿技术论坛's Archiver

Centos 7.4下安装FileBeat