实战篇-Docker 安装ELK(单节点)

作者：码上遇见你2024.03.27 09:35浏览量：8

简介：-

文中的apps目录是在/home/ubuntu目录下

创建docker网络

docker network create -d bridge elastic

拉取elasticsearch 8.4.3版本

docker pull elasticsearch:8.4.3

第一次执行docker脚本

docker run -it \
-p 9200:9200 \
-p 9300:9300 \
--name elasticsearch \
--net elastic \
-e ES_JAVA_OPTS="-Xms1g -Xmx1g" \
-e "discovery.type=single-node" \
-e LANG=C.UTF-8 \
-e LC_ALL=C.UTF-8 \
elasticsearch:8.4.3

注意第一次执行脚本不要加-d这个参数，否则看不到服务首次运行时生成的随机密码和随机 enrollment token

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  L3WKr6ROTiK_DbqzBr8c

ℹ️  HTTP CA certificate SHA-256 fingerprint:
  5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a

ℹ️  Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMTkuMC4yOjkyMDAiXSwiZmdyIjoiNWU3ZDlmZTQ4YzQ4NWMyNzYxZjllN2E5OWI5ZDU3MzdlNGUzNGRjNTViOWJmNjkyOWQ5MjlmYjM0ZDYxYTExYSIsImtleSI6Ik4yMGtkSTRCWDZkeG1BS2lMWGtvOlVPenpCN3dYUUlXV2xmcjZhSTNiQncifQ==

ℹ️ Configure other nodes to join this cluster:
• Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMTkuMC4yOjkyMDAiXSwiZmdyIjoiNWU3ZDlmZTQ4YzQ4NWMyNzYxZjllN2E5OWI5ZDU3MzdlNGUzNGRjNTViOWJmNjkyOWQ5MjlmYjM0ZDYxYTExYSIsImtleSI6Ik9XMGtkSTRCWDZkeG1BS2lMWGtwOmI0Y05razVpUWlPTncwTkMwYWM5akEifQ==

  If you're running in Docker, copy the enrollment token and run:
  `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.4.3`
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

token

eyJ2ZXIiOiI4LjQuMyIsImFkciI6WyIxNzIuMTkuMC4yOjkyMDAiXSwiZmdyIjoiNWU3ZDlmZTQ4YzQ4NWMyNzYxZjllN2E5OWI5ZDU3MzdlNGUzNGRjNTViOWJmNjkyOWQ5MjlmYjM0ZDYxYTExYSIsImtleSI6Ik9XMGtkSTRCWDZkeG1BS2lMWGtwOmI0Y05razVpUWlPTncwTkMwYWM5akEifQ==

创建相应目录并复制配置文件到主机

 mkdir -p apps/elk8.4.3/elasticsearch
# 这个cp命令是在 /home/ubuntu目录下执行的
docker cp elasticsearch:/usr/share/elasticsearch/config apps/elk8.4.3/elasticsearch/

docker cp elasticsearch:/usr/share/elasticsearch/data apps/elk8.4.3/elasticsearch/

docker cp elasticsearch:/usr/share/elasticsearch/plugins apps/elk8.4.3/elasticsearch/

docker cp elasticsearch:/usr/share/elasticsearch/logs apps/elk8.4.3/elasticsearch/

删除容器

docker rm -f elasticsearch

修改apps/elk8.4.3/elasticsearch/config/elasticsearch.yml

vim apps/elk8.4.3/elasticsearch/config/elasticsearch.yml

添加

增加：xpack.monitoring.collection.enabled: true
说明：添加这个配置以后在kibana中才会显示联机状态，否则会显示脱机状态

启动elasticsearch

docker run -it \
-d \
-p 9200:9200 \
-p 9300:9300 \
--name elasticsearch \
--net elastic \
-e ES_JAVA_OPTS="-Xms1g -Xmx1g" \
-e "discovery.type=single-node" \
-e LANG=C.UTF-8 \
-e LC_ALL=C.UTF-8 \
-v /home/ubuntu/apps/elk8.4.3/elasticsearch/config:/usr/share/elasticsearch/config \
-v /home/ubuntu/apps/elk8.4.3/elasticsearch/data:/usr/share/elasticsearch/data \
-v /home/ubuntu/apps/elk8.4.3/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-v /home/ubuntu/apps/elk8.4.3/elasticsearch/logs:/usr/share/elasticsearch/logs \
elasticsearch:8.4.3

启动验证

https://xxxxx:9200/

用户名:elastic
密码在第一次启动时保存下来的信息中查找
Kibana
安装Kibana
```
docker pull kibana:8.4.3
```

启动Kibana

docker run -it \
--restart=always \
--log-driver json-file \
--log-opt max-size=100m \
--log-opt max-file=2 \
--name kibana \
-p 5601:5601 \
--net elastic \
kibana:8.4.3

初始化Kibana鉴权凭证

http://xxxx:5601/?code=878708

在textarea中填入之前elasticsearch生成的相关信息,注意这个token只有30分钟的有效期，如果过期了只能进入容器重置token,进入容器执行 /bin/elasticsearch-create-enrollment-token -s kibana —url “https://127.0.0.1:9200“

输入完token后出现以下界面：

同时服务端会打印出相关的log如图：

kibana验证

将服务端的log中输出的验证码输入到浏览器中，我这里是628503

创建kibana目录并copy相关配置信息

mkdir apps/elk8.4.3/kibana
# 这个cp命令是在 /home/ubuntu目录下执行的
docker cp kibana:/usr/share/kibana/config apps/elk8.4.3/kibana/


docker cp kibana:/usr/share/kibana/data apps/elk8.4.3/kibana/

docker cp kibana:/usr/share/kibana/plugins apps/elk8.4.3/kibana/

docker cp kibana:/usr/share/kibana/logs apps/elk8.4.3/kibana/

sudo chown -R 1000:1000 apps/elk8.4.3/kibana

修改apps/elk8.4.3/kibana/config/kibana.yml

### >>>>>>> BACKUP START: Kibana interactive setup (2024-03-25T07:30:11.689Z)

#
# ** THIS IS AN AUTO-GENERATED FILE **
#

# Default Kibana configuration for docker target
#server.host: "0.0.0.0"
#server.shutdownTimeout: "5s"
#elasticsearch.hosts: [ "http://elasticsearch:9200" ]
#monitoring.ui.container.elasticsearch.enabled: true
### >>>>>>> BACKUP END: Kibana interactive setup (2024-03-25T07:30:11.689Z)

# This section was automatically generated during setup.
i18n.locale: "zh-CN"
server.host: 0.0.0.0
server.shutdownTimeout: 5s
# #这个ip一定是elasticsearch的容器ip，可使用docker inspect | grep -i ipaddress
elasticsearch.hosts: ['https://172.19.0.2:9200']
monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MTEzNTE4MTA5NDM6ZHZ1R3M5cV9RRlc2NmQ3dE9WaWM0QQ
elasticsearch.ssl.certificateAuthorities: [/usr/share/kibana/data/ca_1711351811685.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://172.19.0.2:9200'], ca_trusted_fingerprint: 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a}]

删除容器并重启

docker rm -f kibana

docker run -it \
-d \
--restart=always \
--log-driver json-file \
--log-opt max-size=100m \
--log-opt max-file=2 \
--name kibana \
-p 5601:5601 \
--net elastic \
-v /home/ubuntu/apps/elk8.4.3/kibana/config:/usr/share/kibana/config \
-v /home/ubuntu/apps/elk8.4.3/kibana/data:/usr/share/kibana/data \
-v /home/ubuntu/apps/elk8.4.3/kibana/plugins:/usr/share/kibana/plugins \
-v /home/ubuntu/apps/elk8.4.3/kibana/logs:/usr/share/kibana/logs \
kibana:8.4.3

Logstash

Logstash拉取镜像

docker pull logstash:8.4.3

启动

docker run -it \
-d \
--name logstash \
-p 9600:9600 \
-p 5044:5044 \
--net elastic \
logstash:8.4.3

创建目录并同步配置文件

mkdir apps/elk8.4.3/logstash

# 这个cp命令是在 /home/ubuntu目录下执行的
docker cp logstash:/usr/share/logstash/config apps/elk8.4.3/logstash/ 
docker cp logstash:/usr/share/logstash/pipeline apps/elk8.4.3/logstash/ 

sudo cp -rf apps/elk8.4.3/elasticsearch/config/certs apps/elk8.4.3/logstash/config/certs

sudo chown -R 1000:1000 apps/elk8.4.3/logstash

修改配置apps/elk8.4.3/logstash/config/logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: [ "http://172.19.0.2:9200" ]
xpack.monitoring.elasticsearch.username: "elastic"
# 第一次启动elasticsearch是保存的信息中查找 L3WKr6ROTiK_DbqzBr8c
xpack.monitoring.elasticsearch.password: "L3WKr6ROTiK_DbqzBr8c"
xpack.monitoring.elasticsearch.ssl.certificate_authority: "/usr/share/logstash/config/certs/http_ca.crt"
# 第一次启动elasticsearch是保存的信息中查找 5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a
xpack.monitoring.elasticsearch.ssl.ca_trusted_fingerprint: "5e7d9fe48c485c2761f9e7a99b9d5737e4e34dc55b9bf6929d929fb34d61a11a"

修改配置apps/elk8.4.3/logstash/pipeline/logstash.conf

input {
  beats {
    port => 5044
  }
}


filter {
  date {
        # 因为我的日志里，我的time字段格式是2024-03-14T15:34:03+08:00 ，所以要使用以下两行配置
        match => [ "time", "ISO8601" ]
        target => "@timestamp"
  }
  json {
    source => "message"
  }
  mutate {
    remove_field => ["message", "path", "version", "@version", "agent", "cloud", "host", "input", "log", "tags", "_index", "_source", "ecs", "event"]
  }
}


output {
  elasticsearch {
    hosts => ["https://172.18.0.2:9200"]
    index => "douyin-%{+YYYY.MM.dd}"
    ssl => true
    ssl_certificate_verification => false
    cacert => "/usr/share/logstash/config/certs/http_ca.crt"
    ca_trusted_fingerprint => "第一次启动elasticsearch是保存的信息中查找e924551c1453c893114a05656882eea81cb11dd87c1258f83e6f676d2428f8f2"
    user => "elastic"
    password => "第一次启动elasticsearch是保存的信息中查找UkNx8px1yrMYIht30QUc"
  }
}

删除容器并重新启动

docker rm -f logstash

docker run -it \
-d \
--name logstash \
-p 9600:9600 \
-p 5044:5044 \
--net elastic \
-v /home/ubuntu/apps/elk8.4.3/logstash/config:/usr/share/logstash/config \
-v /home/ubuntu/apps/elk8.4.3/logstash/pipeline:/usr/share/logstash/pipeline \
logstash:8.4.3

Filebeat

Filebeat拉取镜像

sudo docker pull elastic/filebeat:8.4.3

FileBeat启动

docker run -it \
-d \
--name filebeat \
--network host \
-e TZ=Asia/Shanghai \
elastic/filebeat:8.4.3 \
filebeat -e  -c /usr/share/filebeat/filebeat.yml

如有问题微信搜索【码上遇见你】。

免费的Chat GPT可微信搜索【AI贝塔】

好了，本章节到此告一段落。希望对你有所帮助，祝学习顺利。

发表评论

关于作者

被阅读数
被赞数
被收藏数

开发者热搜

实战篇-Docker 安装ELK(单节点)

创建docker网络

拉取elasticsearch 8.4.3版本

第一次执行docker脚本

创建相应目录并复制配置文件到主机

删除容器

修改apps/elk8.4.3/elasticsearch/config/elasticsearch.yml

启动elasticsearch

启动验证

Kibana

安装Kibana

启动Kibana

初始化Kibana鉴权凭证

kibana验证

创建kibana目录并copy相关配置信息

修改apps/elk8.4.3/kibana/config/kibana.yml

删除容器并重启

Logstash

Logstash拉取镜像

启动

创建目录并同步配置文件

修改配置apps/elk8.4.3/logstash/config/logstash.yml

修改配置apps/elk8.4.3/logstash/pipeline/logstash.conf

删除容器并重新启动

Filebeat

Filebeat拉取镜像

FileBeat启动

相关文章推荐

文心一言API接入指南

从 MLOps 到 LMOps 的关键技术嬗变

Sugar BI教你怎么做数据可视化 - 拓扑图，让节点连接信息一目了然

更轻量的百度百舸，CCE Stack 智算版发布

打造合规数据闭环，加速自动驾驶技术研发

LMOps 工具链与千帆大模型平台

发表评论

关于作者

最热文章