Kubernetes環境におけるFluentBitを使用したコンテナログの収集方法
FluentBitで全コンテナの標準出力ログを収集する
- FluentBitの設定マニフェストを作成する
---
# FluentBit設定ファイル
apiVersion: v1
kind: ConfigMap
metadata:
name: log-collector-config
namespace: log-system
labels:
app: fluent-bit
data:
# サービス設定
service.conf: |
[SERVICE]
Flush 5
Log_Level info
Daemon off
Parsers_File custom-parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 3030
@INCLUDE input.conf
@INCLUDE filter.conf
@INCLUDE output.conf
# 入力設定
input.conf: |
[INPUT]
Name tail
Tag kubernetes.*
Path /var/log/containers/*.log
Parser container
DB /var/log/fluent_kube.db
Mem_Buf_Limit 10MB
Skip_Long_Lines On
Refresh_Interval 15
Rotate_Wait 30
# フィルター設定
filter.conf: |
[FILTER]
Name kubernetes
Match kubernetes.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Kube_Tag_Prefix kubernetes.var.log.containers.
Merge_Log On
Merge_Log_Key processed_log
K8S-Logging.Parser On
K8S-Logging.Exclude Off
Labels On
Annotations On
# 出力設定
output.conf: |
[OUTPUT]
Name es
Match *
Host ${ES_HOST}
Port ${ES_PORT}
HTTP_User admin
HTTP_Passwd changeme
Logstash_Prefix cluster-logs
Logstash_DateFormat %Y.%m
Logstash_Format On
Replace_Dots On
Retry_Limit False
Include_Tag_Key On
Tag_Key log_tag
# パーサー設定
custom-parsers.conf: |
[PARSER]
Name nginx_access
Format regex
Regex ^(?<remote_addr>[^ ]*) - (?<remote_user>[^ ]*) \[(?<time_local>[^\]]*)\] "(?<request>[^"]*)" (?<status>[^ ]*) (?<body_bytes_sent>[^ ]*)(?: "(?<http_referer>[^"]*)" "(?<http_user_agent>[^"]*)")?$
Time_Key time_local
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name container
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
Name cri
Format regex
Regex ^(?<timestamp>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
Time_Key timestamp
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
---
# RBAC ClusterRole設定
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: log-reader-role
namespace: log-system
rules:
- apiGroups: [""]
resources:
- namespaces
- pods
- nodes
verbs: ["get", "list", "watch"]
---
# RBAC ClusterRoleBinding設定
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: log-reader-binding
namespace: log-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: log-reader-role
subjects:
- kind: ServiceAccount
name: fluent-bit-sa
namespace: log-system
---
# ServiceAccount設定
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit-sa
namespace: log-system
---
# DaemonSetデプロイ設定
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: log-collector
namespace: log-system
labels:
app: log-collector
version: v2
kubernetes.io/cluster-service: "true"
spec:
selector:
matchLabels:
app: log-collector
template:
metadata:
labels:
app: log-collector
version: v2
kubernetes.io/cluster-service: "true"
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "3030"
prometheus.io/path: /api/v1/metrics/prometheus
spec:
containers:
- name: fluent-bit
image: fluent/fluent-bit:2.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3030
name: http
protocol: TCP
env:
- name: ES_HOST
value: "elasticsearch.logging.svc.cluster.local"
- name: ES_PORT
value: "9200"
resources:
limits:
memory: 300Mi
cpu: 200m
requests:
cpu: 100m
memory: 150Mi
volumeMounts:
- name: var-log
mountPath: /var/log
readOnly: true
- name: var-container-logs
mountPath: /var/lib/docker/containers
readOnly: true
- name: config-volume
mountPath: /fluent-bit/etc/
terminationGracePeriodSeconds: 30
volumes:
- name: var-log
hostPath:
path: /var/log
type: Directory
- name: var-container-logs
hostPath:
path: /var/lib/docker/containers
type: Directory
- name: config-volume
configMap:
name: log-collector-config
serviceAccountName: fluent-bit-sa
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- operator: "Exists"
effect: "NoExecute"
- ネームスペースを作成する
$ kubectl create namespace log-system
namespace/log-system created
- 設定を適用する
$ kubectl apply -f fluentbit-config.yaml
configmap/log-collector-config created
clusterrole.rbac.authorization.k8s.io/log-reader-role created
clusterrolebinding.rbac.authorization.k8s.io/log-reader-binding created
serviceaccount/fluent-bit-sa created
daemonset.apps/log-collector created
- Podの状態を確認する
$ kubectl get pods -n log-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE
log-collector-abcde 1/1 Running 0 5m 10.244.1.45 worker-node-1
log-collector-fghij 1/1 Running 0 5m 10.244.2.67 worker-node-2
- Elasticsearchでインデックスを確認する
インデックスパターン cluster-logs-* を作成し、ログデータが正しく収集されていることを確認する。KibanaのDiscover機能で、各コンテナからの標準出力ログがリアルタイムで表示される。
- ログ収集の動作テスト
テスト用のPodをデプロイしてログ出力を検証：
$ kubectl run log-test --image=busybox --restart=Never -- nslookup kubernetes.default.svc.cluster.local
KibanaでこのPodのログが正しく収集・表示されていることを確認する。