2.5 Filebeat环境搭建-文艺知识分享平台

查询码: 00000201

2.5 Filebeat环境搭建

作者: 文艺范儿于 2025年12月10日 发布在分类 / Elastic Stack / Elastic Stack ,于 2025年12月10日编辑

filebeat

2.5 Filebeat环境搭建

1. 简介

Filebeat 是 Elastic Stack 中的轻量级日志数据收集器，专门用于转发和集中日志数据。它是 Beats 家族的一员，主要用于日志文件的实时收集和传输。

主要特点：

轻量级，资源消耗低
支持多种输入源（文件、标准输入、Redis等）
内置模块支持常见应用（Nginx、MySQL、Redis等）
支持负载均衡和故障转移

Filebeat工作原理

工作流程：

Harvester：逐行读取日志文件内容
Registry：记录文件读取位置，确保数据不丢失
Output：将数据发送到配置的输出目标

2. 安装部署

# 1.下载
[root@204-web ~]# wget https://mirrors.aliyun.com/elasticstack/8.x/yum/8.19.7/filebeat-8.19.7-x86_64.rpm
# 2.安装
[root@204-web ~]# rpm -ivh filebeat-8.19.7-x86_64.rpm 
# 3.查看帮助信息
[root@204-web ~]# filebeat -h
Usage:
  filebeat [flags]
  filebeat [command]

Available Commands:
  export      Export current config or index template
  generate    Generate Filebeat modules, filesets and fields.yml
  help        Help about any command
  keystore    Manage secrets keystore
  modules     Manage configured modules
  run         Run filebeat
  setup       Setup index template, dashboards and ML jobs
  test        Test config
  version     Show current version info

Flags:
  -E, --E setting=value              Configuration overwrite
  -M, --M setting=value              Module configuration overwrite
  -N, --N                            Disable actual publishing for testing
  -c, --c string                     Configuration file, relative to path.config (default "filebeat.yml")
      --cpuprofile string            Write cpu profile to file
  -d, --d string                     Enable certain debug selectors
  -e, --e                            Log to stderr and disable syslog/file output
      --environment environmentVar   set environment being ran in (default default)
  -h, --help                         help for filebeat
      --httpprof string              Start pprof http server
      --memprofile string            Write memory profile to this file
      --modules string               List of enabled modules (comma separated)
      --once                         Run filebeat only once until all harvesters reach EOF
      --path.config string           Configuration path
      --path.data string             Data path
      --path.home string             Home path
      --path.logs string             Logs path
      --plugin pluginList            Load additional plugins
      --strict.perms                 Strict permission checking on config files (default true)
  -v, --v                            Log at INFO level

Use "filebeat [command] --help" for more information about a command.

扩展：多实例安装只需要去配置不同的数据目录即可。

filebeat -e -c c-es.yaml --path.data /tmp/filebeat2

3. 监控nginx日志

a. 架构图

粘贴图片

b. Filebeat Nginx日志采集

一、优化nginx日志格式：

注意：本人使用的nginx是Tengine。原生的nginx不支持自定义日志写入管道。也就是这种写法：pipe:rollback logs/access_log interval=1d baknum=7 maxsize=2G

确保 nginx.conf 中使用的是标准 main 格式（或你自定义但结构清晰的格式）：

# nginx.conf
log_format json_combined escape=json
      
    '{'
      '"@timestamp":"$time_iso8601",'
      '"host":"$server_addr",'
      '"clientip":"$remote_addr",'
      '"SendBytes":$body_bytes_sent,'
      '"responsetime":$request_time,'
      '"upstreamtime":"$upstream_response_time",'
      '"upstreamhost":"$upstream_addr",'
      '"http_host":"$host",'
      '"uri":"$uri",'
      '"domain":"$host",'
      '"xff":"$http_x_forwarded_for",'
      '"referer":"$http_referer",'
      '"tcp_xff":"$proxy_protocol_addr",'
      '"http_user_agent":"$http_user_agent",'
      '"status":"$status"'
    '}'; 

    access_log logs/access.log json_combined;
    access_log  "pipe:rollback logs/access_log interval=1d baknum=7 maxsize=2G" json_combined;
    error_log  logs/error.log crit;
    error_log  "pipe:rollback logs/error_log interval=1d baknum=7 maxsize=2G" crit;：:

#注意修改完nginx记得需要reload生效配置

二、创建索引模版

#!/bin/bash
# create_correct_nginx_template.sh

ES_HOST="10.0.0.91:9200"

echo "创建匹配现有映射的索引模板..."

# 删除可能冲突的模板
curl -X DELETE "http://$ES_HOST/_index_template/nginx-access" 2>/dev/null || true

# 创建基于现有映射的模板
curl -X PUT "http://$ES_HOST/_index_template/nginx-access" \
  -H "Content-Type: application/json" \
  -d '{
    "index_patterns": ["nginx-access-*"],
    "template": {
      "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1,
        "index.refresh_interval": "30s"
      },
      "mappings": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "SendBytes": {
            "type": "long"
          },
          "clientip": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "domain": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "host": {
            "properties": {
              "name": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "http_host": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "http_user_agent": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "log_type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "referer": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "responsetime": {
            "type": "long"
          },
          "status": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "tcp_xff": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "upstreamhost": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "upstreamtime": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "uri": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "xff": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    },
    "priority": 100
  }'

echo "索引模板创建完成"
echo "验证模板:"
curl -s "http://$ES_HOST/_index_template/nginx-access" | jq '.'

三、Filebeat 配置：

# 1.主配置文件/etc/filebeat/filebeat.yml
[root@204-web ~]# cd /etc/filebeat/
[root@204-web filebeat]# cat filebeat.yml 
# filebeat.yml
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/deploy/nginx/logs/access.log
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  fields:
    log_type: "nginx_access"
  fields_under_root: true

# 处理多行日志（如有错误日志需要监控）
- type: log
  enabled: true
  paths:
    - /home/deploy/nginx/logs/error.log
  multiline.pattern: '^[0-9]{4}/[0-9]{2}/[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  fields:
    log_type: "nginx_error"

# ====================== 输出配置 ======================
output.elasticsearch:
  hosts: ["http://10.0.0.91:9200", "http://10.0.0.92:9200", "http://10.0.0.93:9200"]
  indices:
    - index: "nginx-access-%{+yyyy.MM.dd}"
      when.equals:
        log_type: "nginx_access"
    - index: "nginx-error-%{+yyyy.MM.dd}"
      when.equals:
        log_type: "nginx_error"

# ====================== 处理器配置 ======================
processors:
  - drop_fields:
      fields: ["log.offset", "prospector.type", "input.type"]
  - decode_json_fields:
      fields: ["message"]
      process_array: false
      max_depth: 1
      target: ""
      overwrite_keys: false
      
# ====================== 模块配置 ======================
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 10s

# ====================== 监控配置 ======================
monitoring:
  enabled: true
  elasticsearch:
    hosts: ["http://10.0.0.91:9200", "http://10.0.0.92:9200", "http://10.0.0.93:9200"]

# ====================== 日志配置 ======================
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

# ==================== 性能优化配置 ===================
queue:
  mem:
    events: 4096
    flush.min_events: 512
    flush.timeout: 5s
  
# 2.Nginx模块配置
#/etc/filebeat/modules.d/nginx.yml
[root@204-web modules.d]# pwd
/etc/filebeat/modules.d
[root@204-web modules.d]# cat nginx.yml
- module: nginx
  access:
    enabled: true
    var.paths: ["/home/deploy/nginx/logs/access.log"]
    input: 
      fields:
        log_type: "nginx_access"
  error:
    enabled: true
    var.paths: ["/home/deploy/nginx/logs/error.log"]
    input:
      fields:
        log_type: "nginx_error"
"

四、启动Filebeat

# 测试配置
[root@204-web ~]# filebeat test config
[root@204-web ~]# filebeat test output

# 启用Nginx模块
[root@204-web ~]# filebeat modules enable nginx

# 启动
[root@204-web ~]# systemctl daemon-reload
[root@204-web ~]# systemctl enable filebeat
[root@204-web ~]# systemctl start filebeat

# 或 前台调试
[root@204-web ~]# filebeat  -e -c /etc/filebeat/filebeat.yml

五、访问nginx让产生日志

#多来点，或者写脚本循环访问
[root@204-web ~]#while true; do curl 10.0.0.204;sleep 0.5;done

六、Kibana可视化仪表板

1.使用文件导入仪表板和视图

##注意：kibana和filebeat的时间要同步，不然对出现数据时间不一致

把一下json保存文件导入到kibana

点击左侧菜单栏的 ☰ (菜单按钮)-->

选择 Stack Management-->

在Stack Management页面，选择 保存的对象 (Saved Objects)-->

点击右上角的 导入 (Import)-->

点击 选择或拖放文件，选择刚才创建的 nginx-kibana-all.ndjson

确保勾选 自动覆盖所有冲突的保存对象

点击导入

导入完成后，点击完成

##nginx-kibana-all.ndjson
##以下ndjson会自动创建一个名字为nginx-access的数据视图，
#索引模式为nginx-* 时间字段：@timestamp 并且指定高级设置里面的ID为dongshufeng

[root@204-web ~]# vi nginx-kibana-all.ndjson
##注意只有9行
[root@204-web ~]#cat nginx-kibana-all.ndjson
{"attributes":{"allowHidden":false,"fieldAttrs":"{}","fieldFormatMap":"{}","fields":"[]","name":"nginx-access","runtimeFieldMap":"{}","sourceFilters":"[]","timeFieldName":"@timestamp","title":"nginx-*"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T05:54:23.492Z","id":"dongshufeng","managed":false,"references":[],"type":"index-pattern","typeMigrationVersion":"8.0.0","updated_at":"2025-11-28T05:54:23.492Z","version":"WzU3MjcsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"请求量趋势","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"请求量趋势\",\"type\":\"area\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"params\":{\"emptyAsNull\":false},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"params\":{\"field\":\"@timestamp\",\"useNormalizedEsInterval\":true,\"extendToTimeRange\":false,\"scaleMetricValues\":false,\"interval\":\"auto\",\"used_interval\":\"0ms\",\"drop_partials\":false,\"min_doc_count\":1,\"extended_bounds\":{}},\"schema\":\"segment\"}],\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"bottom\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"@timestamp per 30 minutes\"},\"type\":\"category\"}],\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"}},\"legendPosition\":\"right\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"Count\"},\"drawLinesBetweenPoints\":true,\"mode\":\"stacked\",\"show\":\"true\",\"showCircles\":true,\"type\":\"area\",\"valueAxis\":\"ValueAxis-1\"}],\"times\":[],\"type\":\"area\",\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true},\"name\":\"LeftAxis-1\",\"position\":\"left\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Count\"},\"type\":\"value\"}],\"legendSize\":\"auto\",\"detailedTooltip\":true,\"palette\":{\"type\":\"palette\",\"name\":\"default\"},\"fittingFunction\":\"linear\",\"truncateLegend\":true,\"maxLegendLines\":1,\"radiusRatio\":9,\"thresholdLine\":{\"show\":false,\"value\":10,\"width\":1,\"style\":\"full\",\"color\":\"#E7664C\"},\"labels\":{}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:29:08.264Z","id":"8bac3341-0058-459f-bd93-199e7ab3e22a","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:29:08.264Z","version":"WzU3NDMsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"响应时间趋势","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"响应时间趋势\",\"type\":\"line\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"avg\",\"params\":{\"field\":\"responsetime\"},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"params\":{\"field\":\"@timestamp\",\"useNormalizedEsInterval\":true,\"extendToTimeRange\":false,\"scaleMetricValues\":false,\"interval\":\"auto\",\"used_interval\":\"0ms\",\"drop_partials\":false,\"min_doc_count\":1,\"extended_bounds\":{}},\"schema\":\"segment\"}],\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"bottom\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"@timestamp per 30 minutes\"},\"type\":\"category\"}],\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"}},\"legendPosition\":\"right\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"Average responsetime\"},\"drawLinesBetweenPoints\":true,\"interpolate\":\"linear\",\"lineWidth\":2,\"mode\":\"normal\",\"show\":true,\"showCircles\":true,\"type\":\"line\",\"valueAxis\":\"ValueAxis-1\"}],\"times\":[],\"type\":\"line\",\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true},\"name\":\"LeftAxis-1\",\"position\":\"left\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Average\"},\"type\":\"value\"}],\"legendSize\":\"auto\",\"detailedTooltip\":true,\"palette\":{\"type\":\"palette\",\"name\":\"default\"},\"fittingFunction\":\"linear\",\"truncateLegend\":true,\"maxLegendLines\":1,\"labels\":{},\"radiusRatio\":9,\"thresholdLine\":{\"show\":false,\"value\":10,\"width\":1,\"style\":\"full\",\"color\":\"#E7664C\"}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:28:39.091Z","id":"34a9bffa-47b9-405c-ad92-bef0e84e6db2","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:28:39.091Z","version":"WzU3NDEsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"HTTP状态码分布","uiStateJSON":"{\"vis\":{\"colors\":{\"200\":\"#7EB26D\",\"302\":\"#EF843C\",\"304\":\"#E24D42\",\"404\":\"#EAB839\",\"500\":\"#BF1B00\"}}}","version":1,"visState":"{\"title\":\"HTTP状态码分布\",\"type\":\"pie\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"params\":{\"emptyAsNull\":false},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"params\":{\"field\":\"status.keyword\",\"orderBy\":\"1\",\"order\":\"desc\",\"size\":10,\"otherBucket\":false,\"otherBucketLabel\":\"其他\",\"missingBucket\":false,\"missingBucketLabel\":\"缺失\",\"includeIsRegex\":true,\"excludeIsRegex\":true},\"schema\":\"segment\"}],\"params\":{\"addTooltip\":true,\"isDonut\":true,\"labels\":{\"show\":false,\"values\":true},\"legendPosition\":\"right\",\"type\":\"pie\",\"legendDisplay\":\"show\",\"legendSize\":\"auto\",\"nestedLegend\":false,\"truncateLegend\":true,\"maxLegendLines\":1,\"distinctColors\":false,\"emptySizeRatio\":0.3,\"palette\":{\"type\":\"palette\",\"name\":\"default\"}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:26:57.223Z","id":"e20f226c-967c-4fac-88e7-6a291bc33a06","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:26:57.223Z","version":"WzU3MzMsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"数据传输量统计","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"数据传输量统计\",\"type\":\"metric\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"sum\",\"params\":{\"field\":\"SendBytes\",\"emptyAsNull\":false},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"avg\",\"params\":{\"field\":\"SendBytes\"},\"schema\":\"metric\"},{\"id\":\"3\",\"enabled\":true,\"type\":\"max\",\"params\":{\"field\":\"SendBytes\"},\"schema\":\"metric\"}],\"params\":{\"addLegend\":false,\"addTooltip\":true,\"fontSize\":\"60\",\"handleNoResults\":true,\"type\":\"metric\",\"metric\":{\"percentageMode\":false,\"useRanges\":false,\"colorSchema\":\"Green to Red\",\"metricColorMode\":\"None\",\"colorsRange\":[{\"from\":0,\"to\":10000}],\"labels\":{\"show\":true},\"invertColors\":false,\"style\":{\"bgFill\":\"#000\",\"bgColor\":false,\"labelColor\":false,\"subText\":\"\",\"fontSize\":60}}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:27:21.834Z","id":"6ce742f2-040a-4e30-9e15-2cb58b9ac9a1","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:27:21.834Z","version":"WzU3MzUsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"URI请求量Top 10","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"URI请求量Top 10\",\"type\":\"horizontal_bar\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"params\":{\"emptyAsNull\":false},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"params\":{\"field\":\"uri.keyword\",\"orderBy\":\"1\",\"order\":\"desc\",\"size\":10,\"otherBucket\":false,\"otherBucketLabel\":\"其他\",\"missingBucket\":false,\"missingBucketLabel\":\"缺失\",\"includeIsRegex\":true,\"excludeIsRegex\":true},\"schema\":\"segment\"}],\"params\":{\"addLegend\":false,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"left\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{},\"type\":\"category\"}],\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"}},\"legendPosition\":\"right\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"Count\"},\"drawLinesBetweenPoints\":true,\"mode\":\"normal\",\"show\":true,\"showCircles\":false,\"type\":\"histogram\",\"valueAxis\":\"ValueAxis-1\"}],\"times\":[],\"type\":\"horizontal_bar\",\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true},\"name\":\"LeftAxis-1\",\"position\":\"bottom\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Count\"},\"type\":\"value\"}],\"legendSize\":\"auto\",\"detailedTooltip\":true,\"palette\":{\"type\":\"palette\",\"name\":\"default\"},\"truncateLegend\":true,\"maxLegendLines\":1,\"labels\":{},\"radiusRatio\":0,\"thresholdLine\":{\"show\":false,\"value\":10,\"width\":1,\"style\":\"full\",\"color\":\"#E7664C\"}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:27:43.379Z","id":"22dd8c36-6a7b-45d1-9cc0-259bd1edd2ce","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:27:43.379Z","version":"WzU3MzcsMV0="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"客户端IP Top 10","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"客户端IP Top 10\",\"type\":\"horizontal_bar\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"params\":{\"emptyAsNull\":false},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"params\":{\"field\":\"clientip.keyword\",\"orderBy\":\"1\",\"order\":\"desc\",\"size\":10,\"otherBucket\":false,\"otherBucketLabel\":\"其他\",\"missingBucket\":false,\"missingBucketLabel\":\"缺失\",\"includeIsRegex\":true,\"excludeIsRegex\":true},\"schema\":\"segment\"}],\"params\":{\"addLegend\":false,\"addTimeMarker\":false,\"addTooltip\":true,\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"labels\":{\"show\":true,\"truncate\":100},\"position\":\"left\",\"scale\":{\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{},\"type\":\"category\"}],\"grid\":{\"categoryLines\":false,\"style\":{\"color\":\"#eee\"}},\"legendPosition\":\"right\",\"seriesParams\":[{\"data\":{\"id\":\"1\",\"label\":\"计数\"},\"drawLinesBetweenPoints\":true,\"mode\":\"normal\",\"show\":true,\"showCircles\":false,\"type\":\"histogram\",\"valueAxis\":\"ValueAxis-1\"}],\"times\":[],\"type\":\"horizontal_bar\",\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"labels\":{\"filter\":false,\"rotate\":0,\"show\":true},\"name\":\"LeftAxis-1\",\"position\":\"bottom\",\"scale\":{\"mode\":\"normal\",\"type\":\"linear\"},\"show\":true,\"style\":{},\"title\":{\"text\":\"Count\"},\"type\":\"value\"}],\"legendSize\":\"auto\",\"detailedTooltip\":true,\"palette\":{\"type\":\"palette\",\"name\":\"default\"},\"truncateLegend\":true,\"maxLegendLines\":1,\"labels\":{},\"radiusRatio\":0,\"thresholdLine\":{\"show\":false,\"value\":10,\"width\":1,\"style\":\"full\",\"color\":\"#E7664C\"}}}"},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T06:28:14.187Z","id":"d1b7df73-eefb-438f-8e2f-1fbdb53b5b85","managed":false,"references":[{"id":"dongshufeng","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2025-11-28T06:28:14.187Z","version":"WzU3MzgsMV0="}
{"attributes":{"controlGroupInput":{"chainingSystem":"HIERARCHICAL","controlStyle":"oneLine","ignoreParentSettingsJSON":"{\"ignoreFilters\":false,\"ignoreQuery\":false,\"ignoreTimerange\":false,\"ignoreValidations\":false}","panelsJSON":"{}","showApplySelections":false},"description":"Nginx访问日志实时监控仪表板","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"filter\":[],\"query\":{\"language\":\"kuery\",\"query\":\"\"}}"},"optionsJSON":"{\"useMargins\":true,\"syncColors\":true,\"syncCursor\":true,\"syncTooltips\":true,\"hidePanelTitles\":false}","panelsJSON":"[{\"type\":\"visualization\",\"panelRefName\":\"panel_43e4f20b-40e2-4cf6-bcad-72324f6b21bb\",\"embeddableConfig\":{\"savedObjectId\":\"8bac3341-0058-459f-bd93-199e7ab3e22a\"},\"panelIndex\":\"43e4f20b-40e2-4cf6-bcad-72324f6b21bb\",\"gridData\":{\"i\":\"43e4f20b-40e2-4cf6-bcad-72324f6b21bb\",\"y\":0,\"x\":0,\"w\":13,\"h\":8}},{\"type\":\"visualization\",\"panelRefName\":\"panel_a9516538-a7fe-4851-9742-a37d93e7a6be\",\"embeddableConfig\":{\"savedObjectId\":\"34a9bffa-47b9-405c-ad92-bef0e84e6db2\"},\"panelIndex\":\"a9516538-a7fe-4851-9742-a37d93e7a6be\",\"gridData\":{\"i\":\"a9516538-a7fe-4851-9742-a37d93e7a6be\",\"y\":0,\"x\":13,\"w\":10,\"h\":8}},{\"type\":\"visualization\",\"panelRefName\":\"panel_e98fb440-de3d-49ea-acca-9a6a73308787\",\"embeddableConfig\":{\"savedObjectId\":\"e20f226c-967c-4fac-88e7-6a291bc33a06\",\"enhancements\":{\"dynamicActions\":{\"events\":[]}},\"uiState\":{\"vis\":{\"colors\":{\"200\":\"#7EB26D\",\"302\":\"#EF843C\",\"304\":\"#E24D42\",\"404\":\"#EAB839\",\"500\":\"#BF1B00\"}}}},\"panelIndex\":\"e98fb440-de3d-49ea-acca-9a6a73308787\",\"gridData\":{\"i\":\"e98fb440-de3d-49ea-acca-9a6a73308787\",\"y\":0,\"x\":23,\"w\":10,\"h\":8}},{\"type\":\"visualization\",\"panelRefName\":\"panel_3bf36da7-63bd-4f35-8cb5-daa3e98532d6\",\"embeddableConfig\":{\"savedObjectId\":\"6ce742f2-040a-4e30-9e15-2cb58b9ac9a1\"},\"panelIndex\":\"3bf36da7-63bd-4f35-8cb5-daa3e98532d6\",\"gridData\":{\"i\":\"3bf36da7-63bd-4f35-8cb5-daa3e98532d6\",\"y\":17,\"x\":0,\"w\":33,\"h\":8}},{\"type\":\"visualization\",\"panelRefName\":\"panel_5209844f-6dda-43a1-8e04-3f13b5317709\",\"embeddableConfig\":{\"savedObjectId\":\"22dd8c36-6a7b-45d1-9cc0-259bd1edd2ce\"},\"panelIndex\":\"5209844f-6dda-43a1-8e04-3f13b5317709\",\"gridData\":{\"i\":\"5209844f-6dda-43a1-8e04-3f13b5317709\",\"y\":8,\"x\":0,\"w\":16,\"h\":9}},{\"type\":\"visualization\",\"panelRefName\":\"panel_c6b2bc4f-1820-45f7-81b7-cc10cf0990ea\",\"embeddableConfig\":{\"savedObjectId\":\"d1b7df73-eefb-438f-8e2f-1fbdb53b5b85\"},\"panelIndex\":\"c6b2bc4f-1820-45f7-81b7-cc10cf0990ea\",\"gridData\":{\"i\":\"c6b2bc4f-1820-45f7-81b7-cc10cf0990ea\",\"y\":8,\"x\":16,\"w\":17,\"h\":9}}]","timeRestore":false,"title":"Nginx访问监控大盘","version":3},"coreMigrationVersion":"8.8.0","created_at":"2025-11-28T05:52:17.712Z","id":"nginx-monitoring-dashboard","managed":false,"references":[{"id":"dongshufeng","name":"43e4f20b-40e2-4cf6-bcad-72324f6b21bb:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"8bac3341-0058-459f-bd93-199e7ab3e22a","name":"43e4f20b-40e2-4cf6-bcad-72324f6b21bb:panel_43e4f20b-40e2-4cf6-bcad-72324f6b21bb","type":"visualization"},{"id":"dongshufeng","name":"a9516538-a7fe-4851-9742-a37d93e7a6be:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"34a9bffa-47b9-405c-ad92-bef0e84e6db2","name":"a9516538-a7fe-4851-9742-a37d93e7a6be:panel_a9516538-a7fe-4851-9742-a37d93e7a6be","type":"visualization"},{"id":"dongshufeng","name":"e98fb440-de3d-49ea-acca-9a6a73308787:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"e20f226c-967c-4fac-88e7-6a291bc33a06","name":"e98fb440-de3d-49ea-acca-9a6a73308787:panel_e98fb440-de3d-49ea-acca-9a6a73308787","type":"visualization"},{"id":"dongshufeng","name":"3bf36da7-63bd-4f35-8cb5-daa3e98532d6:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"6ce742f2-040a-4e30-9e15-2cb58b9ac9a1","name":"3bf36da7-63bd-4f35-8cb5-daa3e98532d6:panel_3bf36da7-63bd-4f35-8cb5-daa3e98532d6","type":"visualization"},{"id":"dongshufeng","name":"5209844f-6dda-43a1-8e04-3f13b5317709:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"22dd8c36-6a7b-45d1-9cc0-259bd1edd2ce","name":"5209844f-6dda-43a1-8e04-3f13b5317709:panel_5209844f-6dda-43a1-8e04-3f13b5317709","type":"visualization"},{"id":"dongshufeng","name":"c6b2bc4f-1820-45f7-81b7-cc10cf0990ea:kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"},{"id":"d1b7df73-eefb-438f-8e2f-1fbdb53b5b85","name":"c6b2bc4f-1820-45f7-81b7-cc10cf0990ea:panel_c6b2bc4f-1820-45f7-81b7-cc10cf0990ea","type":"visualization"}],"type":"dashboard","typeMigrationVersion":"10.3.0","updated_at":"2025-11-28T06:31:57.305Z","version":"WzU3NDksMV0="}
{"excludedObjects":[],"excludedObjectsCount":0,"exportedCount":8,"missingRefCount":0,"missingReferences":[]}

以上ndjson创建的内容：

粘贴图片

展示：

粘贴图片

2. 手动创建仪表板

1.访问Kibana: http://10.0.0.91:5601

2.进入 Stack Management → Kibana → 数据视图 → 创建数据视图

Name: nginx-access

Index pattern: nginx-*

Timestamp field: @timestamp

再创建可视化组件

Analytics → Dashboard → Create dashboard
Create new → Lens
逐个创建可视化组件

综合
评分

0.0分

0人参与

备注	修改日期	修改人
内容更新	2025-12-10 23:11:01[当前版本]	文艺范儿
创建版本	2025-12-10 23:09:44	文艺范儿

附件类型	PNG
paste1765379106297.png(9048.0b) paste1765379153512.png(14886.0b) paste1765379322591.png(295440.0b) paste1765379362143.png(161583.0b)

[Title]

2.5 Filebeat环境搭建

1. 简介

2. 安装部署

3. 监控nginx日志

a. 架构图

b. Filebeat Nginx日志采集

一、优化nginx日志格式：

二、创建索引模版

三、Filebeat 配置：

四、启动Filebeat

五、访问nginx让产生日志

六、Kibana可视化仪表板

1.使用文件导入仪表板和视图

2. 手动创建仪表板