CentOS 7.2部署最新ELK 5.3

2020-06-16 17:06:56

1、安裝elasticsearch服務

安裝jdk 1.8

rpm -ivh jdk-8u101-linux-x64.rpm
Java -version

設定rpm

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

設定yum源

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

安裝elasticsearch

yum install elasticsearch

設定elasticsearch

cluster.name: htd 設定叢集
node.name: htd-es-1 設定叢集節點
path.data: /home/htd/es-data 設定資料目錄
network.host: 0.0.0.0 設定系結IP
http.port: 9200 設定埠
discovery.zen.ping.unicast.hosts: ["171.16.45.11", "171.16.45.122"] 設定叢集定址
http.cors.enabled: true 設定外掛head存取許可權
http.cors.allow-origin: "*" 設定外掛head存取許可權

mkdir -pv /home/htd/es-data
chmod -R elasticsearch:elasticsearch /home/htd/es-data/

啟動elasticsearch

systemctl start elasticsearch.service
systemctl enable elasticsearch.service
systemctl status elasticsearch.service

測試elasticsearch

存取 http://171.16.45.11:9200
{
"name" : "htd-es-1",
"cluster_name" : "htd",
"cluster_uuid" : "kWIGrN9xTHyzLpOEup9uJA",
"version" : {
"number" : "5.3.0",
"build_hash" : "3adb13b",
"build_date" : "2017-03-23T03:31:50.652Z",
"build_snapshot" : false,
"lucene_version" : "6.4.1"
},
"tagline" : "You Know, for Search"
}

2、安裝logstash服務

安裝jdk 1.8

rpm -ivh jdk-8u101-linux-x64.rpm
java -version

設定rpm

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

設定yum源

安裝logstash

yum install logstash

設定logstash

path.data: /home/htd/logstash 設定資料目錄
path.config: /etc/logstash/conf.d logstash設定目錄
http.host: "0.0.0.0" 設定服務IP
http.port: 9600-9700 設定埠
path.logs: /var/log/logstash logstash紀錄檔目錄

啟動logstash

systemctl start logstash.service
systemctl enable logstash.service
systemctl status logstash.service

設定logstash服務檔案

組態檔目錄：/etc/logstash/conf.d
input {
beats {
port => "5044"
codec => "json"
}
}

filter {
if [type] == "nginx-public"{
geoip {
source => "clientip"
target => "geoip"
database => "/usr/share/logstash/config/GeoLite2-City.mmdb"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float"]
}
}
}

output {
if [type] == "nginx-public" {
elasticsearch {
hosts => ["171.16.45.11:9200"]
index => "logstash-nginx-public-%{+YYYY.MM.dd}"
}
}
}

**==完成組態檔後需要重新啟動logstash或者重新載入組態檔==**

3、安裝kibana服務

安裝jdk 1.8

rpm -ivh jdk-8u101-linux-x64.rpm
java -version

設定rpm

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

設定yum源

安裝kibana

yum install kibana

設定kibana

server.port: 5601 設定埠
server.host: "0.0.0.0" 設定服務地址
server.name: "HTD-Formal-Kibana" 設定kibana服務名
elasticsearch.url: "http://171.16.45.11:9200" 設定連線elasticsearch引數
tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&z={z}' 設定高德地圖

啟動kibana

systemctl enable kibana.service
systemctl start kibana.service
systemctl status kibana.service

存取kibana

http://171.16.45.10:5601/

通過Nginx限制輸入使用者名稱及密碼存取kibana

server {
listen 80;
server_name elk.kibana.htd.cn;
auth_basic "Kibana";
auth_basic_user_file /etc/nginx/htdpasswd;
access_log /home/htd/nginx_logs/kibana_access_80.log htdlog;
error_log /home/htd/nginx_logs/kibana_error_80.log;

location / {
proxy_pass http://htd_kibana;
}
}
設定使用者名稱及密碼：
printf "admin:$(openssl passwd -crypt 123456)n" >/etc/nginx/htdpasswd
測試nginx設定：
nginx -t
過載nginx設定：
nginx -s reload
4、安裝filebeat

下載安裝filebeat

yum install libpcap
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-5.3.0-x86_64.rpm
sudo rpm -vi packetbeat-5.3.0-x86_64.rpm

Nginx紀錄檔採用json

log_format htdlog '{"@timestamp":"$time_iso8601",'
'"host":"$server_addr",'
'"clientip":"$remote_addr",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"upstreamtime":"$upstream_response_time",'
'"upstreamhost":"$upstream_addr",'
'"http_host":"$host",'
'"url":"$uri",'
'"xff":"$http_x_forwarded_for",'
'"referer":"$http_referer",'
'"agent":"$http_user_agent",'
'"status":"$status"}';

設定nginx的filebeat引數

filebeat.prospectors:
- input_type: log
paths:
- /home/htd/nginx_logs/*.log
document_type: "nginx-public"
output.logstash:
hosts: ["171.16.45.13:5044"]

設定tomcat的filebeat引數

filebeat.prospectors:
- input_type: log
paths:
- /home/htd/logs/mallcenter/error.log
document_type: "mallcenter-error"
multiline.pattern: '^[[:space:]]+|^Caused by:' 多行合併
multiline.negate: false
multiline.match: after
output.logstash:
hosts: ["171.16.45.14:5044"]

啟動filebeat

/etc/init.d/filebeat start
/etc/init.d/filebeat stop

基於CentOS 6.9搭建ELK環境指南 http://www.linuxidc.com/Linux/2017-07/145636.htm

Linux紀錄檔分析ELK環境搭建 http://www.linuxidc.com/Linux/2017-07/145494.htm

本文永久更新連結地址：http://www.linuxidc.com/Linux/2017-09/147011.htm

CentOS 7.2部署最新ELK 5.3

熱門文章