logstash7.8.0 and Kibana7.8.0

IT
IT
IT
335
文章
1
评论
2020年7月3日10:43:50 评论 1,837 4048字阅读13分29秒

安装logstash

  • 配置yum源
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# vim /etc/yum.repos.d/logstash.repo
[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
  • 安装
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# yum install logstash -y
  • 或者下载rpm包
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# wget https://artifacts.elastic.co/downloads/logstash/logstash-7.8.0.rpm
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# rpm -ivh logstash-7.8.0.rpm
  • 将执行路径写入环境变量
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# vim /etc/profile
PATH=$PATH:/usr/share/logstash/bin
export PATH
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# source /etc/profile
[root@iZ2ze2jlupmjlwwfuyg30gZ ~]# logstash -V
logstash 7.8.0

Logstash-条件判断

比较操作符:
相等: ==, !=, <, >, <=, >=
正则: =~(匹配正则), !~(不匹配正则)
包含: in(包含), not in(不包含)
布尔操作符:
and(与), or(或), nand(非与), xor(非或)
一元运算符:
!(取反)
()(复合表达式), !()(对复合表达式结果取反)

配置日志拉去路径

[root@iZ2ze2jlupmjlwwfuyg30iZ ~]#  cd /etc/logstash/conf.d/
[root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]# vim logstash-to-es.conf
input {
    file {
        path => ["/home/xiaoxin/app/c-user-center/log/stdout.log"]
        type => "system"
        tags => ["c-user-center"]
        start_position => "beginning"
    }
    file {
        path => ["/home/xiaoxin/app/order-status-sync/log/stdout.log"]
        type => "system"
        tags => ["order-status-sync"]
        start_position => "beginning"
    }
}
filter {
 
}
output {
    if [type] == "system" {
        if [tags][0] == "c-user-center" {
            elasticsearch {
                hosts  => ["http://10.10.0.252:9200","http://10.10.0.253:9200","http://10.10.0.254:9200"]
                index  => "c-user-center-%{+YYYY.MM.dd}"
            }
            stdout { codec=> rubydebug }
        }
        else if [tags][0] == "order-status-sync" {
            elasticsearch {
                hosts  => ["http://10.10.0.252:9200","http://10.10.0.253:9200","http://10.10.0.254:9200"]
                index  => "order-status-sync-%{+YYYY.MM.dd}"
            }
            stdout { codec=> rubydebug }
        }
    }
}
  • 测试是否正确(测试配置文件正确性,如下:-t表示测试,-f 指定文件位置)
[root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]# logstash -t -f /etc/logstash/conf.d/logstash-to-es.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2020-07-03 17:53:28.589 [main] writabledirectory - Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[INFO ] 2020-07-03 17:53:28.611 [main] writabledirectory - Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[WARN ] 2020-07-03 17:53:29.100 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-07-03 17:53:30.942 [LogStash::Runner] Reflections - Reflections took 66 ms to scan 1 urls, producing 21 keys and 41 values
Configuration OK
[INFO ] 2020-07-03 17:53:32.345 [LogStash::Runner] runner - Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
  • 直接运行,加-r参数
[root@iZ2ze2jlupmjlwwfuyg30iZ conf.d]#  logstash -r -f /etc/logstash/conf.d/logstash-to-es.conf
  • 启动服务
[root@iZ2ze2jlupmjlwwfuyg30iZ ~]# systemctl start logstash
[root@iZ2ze2jlupmjlwwfuyg30iZ ~]# systemctl enable logstash
Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.

Elasticsearch上查看

logstash7.8.0 and Kibana7.8.0

安装Kibana

  • 在/etc/yum.repos.d/下新建kibana.repo  配置YUM源地址 内容如下:
[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
[root@elk-es1 ~]# yum install -y kibana
  • 或者使用rpm包
[root@elk-es1 ~]# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.8.0-x86_64.rpm
[root@elk-es1 ~]# rpm -ivh kibana-7.8.0-x86_64.rpm
  • 加入es集群任意节点
[root@elk-es1 ~]# vim /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://10.10.0.252:9200"]
  • 启动服务
[root@elk-es1 ~]# systemctl start kibana.service
[root@elk-es1 ~]# systemctl enable kibana.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.
  • 访问http://ip:5601/

logstash7.8.0 and Kibana7.8.0logstash7.8.0 and Kibana7.8.0

  • 可以看到有在es中生成的索引

logstash7.8.0 and Kibana7.8.0

logstash7.8.0 and Kibana7.8.0

  • 不使用时间戳过滤

logstash7.8.0 and Kibana7.8.0

logstash7.8.0 and Kibana7.8.0

  • 查看拉去过来的日志

logstash7.8.0 and Kibana7.8.0

  • 根据所需,选择对应的日志索引即可。

logstash7.8.0 and Kibana7.8.0

 

 

 

 

 

下载信息 rpm安装包
网盘密码:登录可见
下载地址:登录可见
继续阅读
  • 我的QQ
  • QQ扫一扫
  • weinxin
  • 我的头条
  • 头条扫一扫
  • weinxin
IT
  • 本文由 发表于 2020年7月3日10:43:50
  • 除非特殊声明,本站文章均为原创,转载请务必保留本文链接
Eureka服务剔除下线 ELK

Eureka服务剔除下线

格式 curl -X PUT "http://ip:port/eureka/apps/{application}/{instance}/status?value=OUT_OF_SERVICE" 示例 ...