Visualizing NGINX access logs in Kibana is one of my most visited post in my blog. It is time for a major update. This guide can easily be added into a central log server where someone already collects logs of Docker containers. Especially because it is quite common to run web servers in containerized systems. This tutorial shows you how to parse access logs of NGINX or Apache with syslog-ng and create ECS compatible data in Elasticsearch. I also describe how visualizing NGINX access logs in Kibana can be done.
Tag: Central log server
Simplified guide to logging Containers to Elasticsearch in 2020 (with syslog-ng)
A simplified guide to logging Docker to Elasticsearch. Although there are many tutorials about how to ship Containers logs to Elasticsearch, this one is different from all as it uses syslog-ng. It also works with Podman!
Update: I moved the chapters about parsing and visualizing NGINX / Apache access logs in Kibana into a dedicated post / github repo.
Update 2: This post has been refactored and simplified to be compatible with Elasticsearch ECS and make it easier to implement. Compatible with Elasticsearch 7.x.
Update 3 (2020): Add support both for Docker and Podman. Improved readability. Continue reading Simplified guide to logging Containers to Elasticsearch in 2020 (with syslog-ng)
Creating a central syslog server
Your home network might already contain some devices or systems like a home server, a WiFi router, a media player, or home automation system. It is a best practice creating a central syslog server and storing logs of various sources in one place.
Update: The fail2ban and GeoIP related contents have been merged into post visualizing Fail2ban logs in Kibana.
This post will cover the basics. Creating a central log server and receiving logs from an OpenWRT device. Please note that you can do many more. See the other posts I created in this subject.
- Storing local logs of host operating system,
- Storing logs from OpenWRT based network devices,
- Parsing Fail2ban logs to get GeoIP data,
- To logging Docker to Elasticsearch,