安装logstash
- 配置yum源
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch [root@iZ2ze2jlupmjlwwfuyg30gZ ~]# vim /etc/yum.repos.d/logstash.repo [logstash-7.x] name=Elastic repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md
- 安装
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# yum install logstash -y
- 或者下载rpm包
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# wget https://artifacts.elastic.co/downloads/logstash/logstash-7.8.0.rpm [root@iZ2ze2jlupmjlwwfuyg30gZ ~]# rpm -ivh logstash-7.8.0.rpm
- 将执行路径写入环境变量
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# vim /etc/profile PATH=$PATH:/usr/share/logstash/bin export PATH [root@iZ2ze2jlupmjlwwfuyg30gZ ~]# source /etc/profile [root@iZ2ze2jlupmjlwwfuyg30gZ ~]# logstash -V logstash 7.8.0
Logstash-条件判断
比较操作符:
相等: ==, !=, <, >, <=, >=
正则: =~(匹配正则), !~(不匹配正则)
包含: in(包含), not in(不包含)
布尔操作符:
and(与), or(或), nand(非与), xor(非或)
一元运算符:
!(取反)
()(复合表达式), !()(对复合表达式结果取反)
配置日志拉去路径
[root@iZ2ze2jlupmjlwwfuyg30iZ ~]# cd /etc/logstash/conf.d/ [root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]# vim logstash-to-es.conf input { file { path => ["/home/xiaoxin/app/c-user-center/log/stdout.log"] type => "system" tags => ["c-user-center"] start_position => "beginning" } file { path => ["/home/xiaoxin/app/order-status-sync/log/stdout.log"] type => "system" tags => ["order-status-sync"] start_position => "beginning" } } filter { } output { if [type] == "system" { if [tags][0] == "c-user-center" { elasticsearch { hosts => ["http://10.10.0.252:9200","http://10.10.0.253:9200","http://10.10.0.254:9200"] index => "c-user-center-%{+YYYY.MM.dd}" } stdout { codec=> rubydebug } } else if [tags][0] == "order-status-sync" { elasticsearch { hosts => ["http://10.10.0.252:9200","http://10.10.0.253:9200","http://10.10.0.254:9200"] index => "order-status-sync-%{+YYYY.MM.dd}" } stdout { codec=> rubydebug } } } }
- 测试是否正确(测试配置文件正确性,如下:-t表示测试,-f 指定文件位置)
[root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]# logstash -t -f /etc/logstash/conf.d/logstash-to-es.conf WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [INFO ] 2020-07-03 17:53:28.589 [main] writabledirectory - Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"} [INFO ] 2020-07-03 17:53:28.611 [main] writabledirectory - Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"} [WARN ] 2020-07-03 17:53:29.100 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified [INFO ] 2020-07-03 17:53:30.942 [LogStash::Runner] Reflections - Reflections took 66 ms to scan 1 urls, producing 21 keys and 41 values Configuration OK [INFO ] 2020-07-03 17:53:32.345 [LogStash::Runner] runner - Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
- 直接运行,加-r参数
[root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]# logstash -r -f /etc/logstash/conf.d/logstash-to-es.conf
- 启动服务
[root@iZ2ze2jlupmjlwwfuyg30iZ ~]# systemctl start logstash [root@iZ2ze2jlupmjlwwfuyg30iZ ~]# systemctl enable logstash Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.
Elasticsearch上查看
安装Kibana
- 在/etc/yum.repos.d/下新建kibana.repo 配置YUM源地址 内容如下:
[kibana-7.x] name=Kibana repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md [root@elk-es1 ~]# yum install -y kibana
- 或者使用rpm包
[root@elk-es1 ~]# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.8.0-x86_64.rpm [root@elk-es1 ~]# rpm -ivh kibana-7.8.0-x86_64.rpm
- 加入es集群任意节点
[root@elk-es1 ~]# vim /etc/kibana/kibana.yml server.port: 5601 server.host: "0.0.0.0" elasticsearch.hosts: ["http://10.10.0.252:9200"]
- 启动服务
[root@elk-es1 ~]# systemctl start kibana.service [root@elk-es1 ~]# systemctl enable kibana.service Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.
- 访问http://ip:5601/
- 可以看到有在es中生成的索引
- 不使用时间戳过滤
- 查看拉去过来的日志
- 根据所需,选择对应的日志索引即可。
继续阅读
- 我的QQ
- QQ扫一扫
-
- 我的头条
- 头条扫一扫
-
评论