00000199
节点规划：
| 节点 | 主机名 | 外网IP地址 | 内网IP地址 | 角色 |
|---|---|---|---|---|
| es-node01 |
elk-91 |
10.0.0.91 |
172.16.1.91 |
Master, Data, Ingest |
| es-node02 |
elk-92 |
10.0.0.92 |
172.16.1.92 |
Master, Data, Ingest |
| es-node03 |
elk-93 |
10.0.0.93 |
172.16.1.93 |
Master, Data, Ingest |
ES集群常见术语：
| 术语 | 全称 / 解释 | SRE 关注点与实战意义 |
|---|---|---|
Cluster |
集群 |
由一个或多个节点组成，共同存储数据、提供搜索/写入能力。具备统一的集群名称（ |
Node |
节点 |
单个 ES 实例（进程），可承担多种角色（见下文）。节点发现依靠 |
Master Node |
主节点 |
负责集群状态管理（元数据变更、索引创建删除、分片分配决策）。生产环境必须配置专用 master-eligible 节点，避免与数据节点竞争资源。 |
Data Node |
数据节点 |
存储分片数据、执行 CRUD、搜索、聚合等操作。资源消耗主要在 CPU、内存、磁盘 I/O。 |
Ingest Node |
预处理节点 |
在数据写入前执行 Pipeline 转换（如 Grok、Date、GeoIP）。可单独部署或与其他角色合并。 |
Coordinating Node |
协调节点 |
接收客户端请求，转发到相应 shard，并合并结果返回。默认所有节点都是协调节点，但生产可设专用以分担负载。 |
Index |
索引 |
类似关系型数据库的表，是文档的逻辑集合。索引名必须小写，支持日期模式（ |
Shard |
分片 |
Index 的物理拆分单元，分为 Primary Shard（主分片）与 Replica Shard（副本分片）。分片是水平扩展与并行搜索的基本单位。 |
Primary Shard |
主分片 |
写入入口，负责数据持久化与副本同步。数量在创建 Index 时固定，不可更改（需重建索引）。 |
Replica Shard |
副本分片 |
主分片的拷贝，提供高可用与读负载分担。数量可动态调整。 |
Cluster State |
集群状态 |
包含元数据：节点列表、索引→分片映射、分片分配、模板、别名等。由 Master 节点维护并广播。 |
Metadata |
元数据 |
关于索引、映射、别名、模板的信息。存储在 Cluster State 中，变更需 Master 协调。 |
Index Template |
索引模板 |
预定义索引的 Settings/Mappings，新建匹配的索引时自动套用。 |
Component Template |
组件模板 |
ES 7.8+ 引入，可复用的模板片段，与 Index Template 组合使用。 |
ILM (Index Lifecycle Management) |
索引生命周期管理 |
自动化 Hot→Warm→Cold→Delete 阶段迁移与策略执行。 |
# 1.系统优化 所有节点都要配置
# 配置系统限制
[root@elk-91 ~]# tee /etc/security/limits.conf <<EOF
# Elasticsearch 系统限制配置
* soft nofile 65536
* hard nofile 65536
* soft nproc 4096
* hard nproc 4096
EOF
# 配置虚拟内存
[root@elk-91 ~]# vim /etc/sysctl.conf
...
# Elasticsearch 内核参数
vm.max_map_count=262144
net.core.somaxconn=2048
vm.swappiness=1
...
# 应用配置
[root@elk-91 ~]# sysctl -p
# 2.在三个节点都配置hosts，便于分发管理
vim /etc/hosts
[root@elk-91 ~]# vim /etc/hosts
...
172.16.1.91 es-node01
172.16.1.92 es-node02
172.16.1.93 es-node03
...
# 3.所有节点下载软件包并安装
wget https://mirrors.aliyun.com/elasticstack/8.x/yum/8.19.7/elasticsearch-8.19.7-x86_64.rpm
rpm -ivh elasticsearch-8.19.7-x86_64.rpm
# 4.jvm内存调整，所有节点都根据实际情况配置，原来配置文件在-Xms4g和-Xmx4g位置去掉#即可
[root@elk-93 ~]# vim /etc/elasticsearch/jvm.options
[root@elk-93 ~]# egrep -v "^#|^$" /etc/elasticsearch/jvm.options
-Xms1g
-Xmx1g
-XX:+UseG1GC
-Djava.io.tmpdir=${ES_TMPDIR}
20-:--add-modules=jdk.incubator.vector
23:-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.setAsTypeCache
23:-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.asTypeUncached
-Dorg.apache.lucene.store.MMapDirectory.sharedArenaMaxPermits=1
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
-XX:ErrorFile=hs_err_pid%p.log
-Xlog:gc*,gc+age=trace,safepoint:file=gc.log:utctime,level,pid,tags:filecount=32,filesize=64m
注意节点2和3只修改node.name为es-node02/es-node03，其余一致。
[root@elk-91 ~]# vi /etc/elasticsearch/elasticsearch.yml [root@elk-91 ~]# cat /etc/elasticsearch/elasticsearch.yml # ======================== 集群配置 ======================== cluster.name: es-wyasw node.name: es-node01 # 节点角色配置 node.roles: [ master, data, data_content, data_hot, ingest, ml, remote_cluster_client ] # ======================== 路径配置 ======================== path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch path.repo: /var/lib/elasticsearch/snapshots # ======================== 网络配置 ======================== network.host: 0.0.0.0 http.port: 9200 transport.port: 9300 # ======================== 发现和集群形成 ======================== # 集群发现配置 discovery.seed_hosts: ["es-node01", "es-node02", "es-node03"] # 初始主节点列表 cluster.initial_master_nodes: ["es-node01", "es-node02", "es-node03"] # ======================== 安全配置 ======================== xpack.security.enabled: true # HTTPS配置 xpack.security.http.ssl.enabled: true xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/http.crt xpack.security.http.ssl.key: /etc/elasticsearch/certs/http.key # 传输层SSL xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/http.crt xpack.security.transport.ssl.key: /etc/elasticsearch/certs/http.key xpack.security.transport.ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca.crt"] # ======================== 其他配置 ======================== # 跨域配置 http.cors.enabled: true http.cors.allow-credentials: true http.cors.allow-origin: "*" http.cors.allow-headers: X-Requested-With,Content-Type,Content-Length,Authorization
# 注意一定要配置hosts主机名 cd /usr/share/elasticsearch/ # 生成CA证书（PEM格式，无密码） ./bin/elasticsearch-certutil ca --pem --out /tmp/elastic-ca.zip unzip -o /tmp/elastic-ca.zip -d /tmp/elastic-ca/ # 为节点生成证书 ./bin/elasticsearch-certutil cert --ca-cert /tmp/elastic-ca/ca/ca.crt --ca-key /tmp/elastic-ca/ca/ca.key --name "es-wyasw" --ip "10.0.0.91,10.0.0.92,10.0.0.93" --dns "es-node01,es-node02,es-node03,localhost" --pem --out /tmp/elastic-certs.zip unzip -o /tmp/elastic-certs.zip -d /tmp/elastic-certs/ # 复制证书文件 cp /tmp/elastic-ca/ca/ca.crt /etc/elasticsearch/certs/ cp /tmp/elastic-certs/es-wyasw/es-wyasw.crt /etc/elasticsearch/certs/http.crt cp /tmp/elastic-certs/es-wyasw/es-wyasw.key /etc/elasticsearch/certs/http.key # 设置权限 chown -R elasticsearch:elasticsearch /etc/elasticsearch chmod 644 /etc/elasticsearch/certs/*.crt chmod 600 /etc/elasticsearch/certs/*.key # 验证证书 ls -la /etc/elasticsearch/certs/
# 分发到其他节点（在 es-node-1 上执行） [root@elk-91 elasticsearch]# cd [root@elk-91 ~]# scp -r /etc/elasticsearch/certs es-node02:/etc/elasticsearch/ [root@elk-91 ~]# scp -r /etc/elasticsearch/certs es-node03:/etc/elasticsearch/ # 设置权限在node02和node03 [root@elk-91 ~]# ssh es-node02 "chown -R elasticsearch:elasticsearch /etc/elasticsearch && chmod 644 /etc/elasticsearch/certs/*.crt && chmod 600 /etc/elasticsearch/certs/*.key" [root@elk-91 ~]# ssh es-node03 "chown -R elasticsearch:elasticsearch /etc/elasticsearch && chmod 644 /etc/elasticsearch/certs/*.crt && chmod 600 /etc/elasticsearch/certs/*.key"
#所有节点上执行 systemctl daemon-reexec systemctl enable elasticsearch systemctl start elasticsearch ##如果启动不起来则看日志解决 # 查看启动日志 journalctl -u elasticsearch -f # 或者直接查看日志文件 tail -f /var/log/elasticsearch/wyasw.log
##这种方式也可以用来重置用户密码 # 交互式生成用户密码，用其中一个即可 输入：123456 [root@elk-91 ~]# cd /usr/share/elasticsearch [root@elk-91 elasticsearch]# ./bin/elasticsearch-reset-password -i -u elastic This tool will reset the password of the [elastic] user. You will be prompted to enter the password. Please confirm that you would like to continue [y/N]y Enter password for [elastic]: ##输入密码 Re-enter password for [elastic]: ##再次输入密码 Password for the [elastic] user successfully reset. # 自动生成用户密码 [root@elk-91 ~]# cd /usr/share/elasticsearch [root@elk-91 elasticsearch]# ./bin/elasticsearch-reset-password -u elastic --auto This tool will reset the password of the [elastic] user to an autogenerated value. The password will be printed in the console. Please confirm that you would like to continue [y/N]y Password for the [elastic] user successfully reset. New value: zWBqQcCbfgX2yrlA_xNe
##输入密码123456后应返回以下状态
#-k 表示跳过 SSL 证书验证（因自签证书）。生产环境应配置可信证书。
[root@elk-93 ~]# curl -k -u elastic https://localhost:9200/_cluster/health?pretty
Enter host password for user 'elastic':
{
"cluster_name" : "es-wyasw",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 3,
"active_shards" : 6,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"unassigned_primary_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
##查看节点信息
[root@elk-93 ~]# curl -k -u elastic https://es-node01:9200/_cat/nodes?v
Enter host password for user 'elastic':
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.0.0.92 40 97 2 0.00 0.05 0.06 dhilmrs * es-node02
10.0.0.91 40 95 1 0.01 0.05 0.06 dhilmrs - es-node01
10.0.0.93 37 94 2 0.00 0.05 0.05 dhilmrs - es-node03
##查看集群设置
[root@elk-93 ~]# curl -k -u elastic https://es-node01:9200/_cluster/settings?pretty
Enter host password for user 'elastic':
{
"persistent" : { },
"transient" : { }
}
所有节点修改配置文件并重启服务，后面想用那个可以取消注释修改即可：
[root@elk-91 ~]# vim /etc/elasticsearch/elasticsearch.yml # ======================== 集群配置 ======================== cluster.name: es-wyasw node.name: es-node01 # 节点角色配置 node.roles: [ master, data, data_content, data_hot, ingest, ml, remote_cluster_client ] # ======================== 路径配置 ======================== path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch path.repo: /var/lib/elasticsearch/snapshots # ======================== 网络配置 ======================== network.host: 0.0.0.0 http.port: 9200 transport.port: 9300 # ======================== 发现和集群形成 ======================== # 集群发现配置 discovery.seed_hosts: ["es-node01", "es-node02", "es-node03"] # 初始主节点列表 cluster.initial_master_nodes: ["es-node01", "es-node02", "es-node03"] # ======================== 安全配置 ======================== #安全功能关闭，关闭用户名密码验证 xpack.security.enabled: false # HTTPS配置 # 完全禁用http ssl xpack.security.http.ssl.enabled: false #xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/http.crt #xpack.security.http.ssl.key: /etc/elasticsearch/certs/http.key # 传输层SSL 完全禁用传输层ssl（节点点通信） xpack.security.transport.ssl.enabled: false #xpack.security.transport.ssl.verification_mode: certificate #xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/http.crt #xpack.security.transport.ssl.key: /etc/elasticsearch/certs/http.key #xpack.security.transport.ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca.crt"] # ======================== 其他配置 ======================== # 跨域配置 #http.cors.enabled: true #http.cors.allow-credentials: true #http.cors.allow-origin: "*" #http.cors.allow-headers: X-Requested-With,Content-Type,Content-Length,Authorization ###修改后的配置文件 [root@elk-91 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml cluster.name: es-wyasw node.name: es-node01 node.roles: [ master, data, data_content, data_hot, ingest, ml, remote_cluster_client ] path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch path.repo: /var/lib/elasticsearch/snapshots network.host: 0.0.0.0 http.port: 9200 transport.port: 9300 discovery.seed_hosts: ["es-node01", "es-node02", "es-node03"] cluster.initial_master_nodes: ["es-node01", "es-node02", "es-node03"] xpack.security.enabled: false xpack.security.http.ssl.enabled: false xpack.security.transport.ssl.enabled: false ##node02和node03节点，只修改了node.name为es-node02和es-node03，此处不做示例，配置完成以后重启服务
安装插件过程忽略
插件地址：https://github.com/mobz/elasticsearch-head
查看日志
PNG