Kibana dashboard for home network traffic

Monitor home network traffic with OpenWRT and Syslog-ng

(Last Updated On: 05/08/2020)

I wanted to see what happens on my home network. Is there something going on I should be aware of? Is there any device which creates suspicious connections like phoning home? I will use OpenWRT and syslog-ng to get the answers and Elasticsearch to get analytics.Kibana dashboard showing data from OpenWRT traffic syslogs

SOHO routers are usually not really resourceful, neither is mine. Therefore I needed a solution using as little resource as possible but still capable to answers that questions. My solution uses connection tracking data from the main OpenWRT router. Offload the information from OpenWRT to a central syslog server. Enrich it with GeoIP, Reverse DNS and session length metadata by using syslog-ng. Then analyze the logs with Elasticsearch.

The first part of this blog series answers where the packets come and go and some metrics. What are inside the packets is up to another posts.

Logging connection tracking data with OpenWRT and syslog-ng

My original idea was to log the SYN and ACK,FIN packets with Iptables on the FORWARD chain and correlate them. However it did not work as I planned. Although the most important data are included in syslog messages like network source, destination, port numbers. However the logs cannot be easily correlated to each other to get session data because there is no common identifier in iptables logs which would be unique to any given connection. (Stream ID would be but it is encoded in TCP options.) Logging all packets would simply kill the performance so it does not worth it. I needed an alternative solution.

Founding an essay about Flow-based network accounting in Linux did turn the tide. I realized that ‘nf_conntrack’ in Linux kernel (netfilter) actually keeps track of every connections throughout their lifetime (even UDP which is stateless). I only needed a tool to get that data off the OpenWRT router possibly to syslog-ng. The essay mentioned many tools but ulogd looked the most promising.

Ulogd is capable to log connection tracking data to local syslog. The following example shows a NEW and a DESTROY event of a specific connection logged by ulogd.

Mar 13 15:03:57 openwrt ulogd[21765]: [NEW] ORIG: SRC= DST= PROTO=TCP SPT=57534 DPT=443 PKTS=0 BYTES=0 , REPLY: SRC= DST= PROTO=TCP SPT=443 DPT=57534 PKTS=0 BYTES=0
Mar 13 15:09:00 openwrt ulogd[21765]: [DESTROY] ORIG: SRC= DST= PROTO=TCP SPT=57534 DPT=443 PKTS=9 BYTES=3371 , REPLY: SRC= DST= PROTO=TCP SPT=443 DPT=57534 PKTS=8 BYTES=1301

Note: represents a website, while represents the public IP of my home network.

Configuring ulogd on OpenWRT to send conntrack events to syslog-ng

My OpenWRT systems already send their syslog to a remote central syslog server. OpenWRT uses logread to send syslogs remotely. The remote server runs on syslog-ng. Therefore I only have to configure ulogd to send the connection tracking events to local syslog instead of a file.

Fortunately ulogd can send the events to many destinations. I found a post about logging connection tracking events with ulogd and it helped me to configure the service properly.
On the following link you can find the complete configuration of ulogd I created for OpenWRT 18.06. Nevertheless I describe the details below as well.

  1. First you have to install ulogd and some of its modules. You can do this either in LuCI or in the command line.
    [email protected]:~# opkg update \
    && opkg install ulogd ulogd-mod-nfct ulogd-mod-syslog ulogd-mod-extra
  2. The configuration of ulogd uses an INI style syntax. Two sections will be important for us, [global] and [ct1].
    In [global] section after the list of plugins there will be presets of stacks. Stacks are a list of plugins and they work like commands piped together. There are input plugins, filter plugins and output plugins.
  3. Look for the comment below. We are going to adjust the stack belonging to that comment like this.
    # this is a stack for flow-based logging via LOGEMU
  4. Look for the section called [ct1]. We are adding a new configuration element called hash_enable. Disabling hashes makes ulogd to log both NEW and DESTROY events separately. Otherwise it would only log DESTROY events. Although DESTROY contains everything we need the NEW events as well because of their timestamps, because we will make use of them for building session metadata.
  5. You can do a manual check of the configuration by starting ulogd manually.
    [email protected]:~# ulogd -v
    Mon Mar 11 15:42:51 2019 <5> ulogd.c:843 building new pluginstance stack: 'ct1:NFCT,ip2str1:IP2STR,print1:PRINTFLOW,sys1:SYSLOG'
    Mon Mar 11 15:42:51 2019 <5> ulogd_inpflow_NFCT.c:1399 NFCT plugin working in event mode
  6. The last step is to enable the service and start it.
    [email protected]:~# /etc/init.d/ulogd enable
    [email protected]:~# /etc/init.d/ulogd start

That is all you have to do to make OpenWRT send its connection tracking events to syslog-ng.

Update: Note for general Linux users. I found that netfilter accounting is usually disabled in kernels, especially on OpenSUSE Leap. This will result that PKTS and BYTES will always be 0. To overcome this one must enable accounting by executing the following command.

# runtime only
[email protected]:~# echo "1"> /proc/sys/net/netfilter/nf_conntrack_acct
# make it permanent across system reboots
[email protected]:~# echo 'net.netfilter.nf_conntrack_acct=1' > /etc/sysctl.d/100-custom.conf

Thanks to Eric Leblond’s great blog article for mentioning it.

Processing ulogd log messages from OpenWRT with syslog-ng

Note: the complete syslog-ng configuration can be found on GitHub.

My goal is to parse and enrich the log messages with metadata. Syslog-ng provides many parsers to use out of the box. You can see a short overview about the parsers I use in this case.

Metadata we add to logs Syslog-ng parsers providing the metadata
separate upstream and downstream metrics csv-parser()
parsing key-value pairs from all streams kv-parser()
creating session start, session end and length grouping-by()
GeoIP metadata geoip2()
Reverse DNS for clients and servers python()

Some parsers can be chained together in a single parser{} block. They behave like commands piped together in a Linux shell. One parser’s output will feed the input of the next parser. Thus their order is important.

The first block of parsers look like this. Please refer to the ulogd log samples to get an idea why I do these. I also added comments to the config for clarification.

        parser {
            # Split a single log line into two parts around the comma character
            # Name the first part as ORIG and the second as REPLY
                columns(ORIG, REPLY)
            # Parse the part of the log message available in macro ${ORIG}
            # the separator is the = character
            # Prefix it with "nf_orig."
            # Do the same with the other part of the log message
            # Lookup geoip data for the destination (server)
                prefix( "geo." )
                database( "/etc/syslog-ng/GeoLite2-City.mmdb" )

Further parsers like grouping-by() and python() will be discussed later.

Correlating log messages from OpenWRT with syslog-ng

The central syslog server receives two type of log messages from each connection. One message from NEW events and another from DESTROY events. These two symbolize the beginning and the end of a session. I will use grouping-by() parser to correlate these messages into one context and get the session length metadata. The admin guide has a flow chart about how messages are added to the context and how it gets terminated. You may want to read that in advance.

The parser uses key() and scope() to build-up a context and identify which messages needs to be added to the context.

For specifying key() requires already parsed data. My setup can be translated to this: “Messages containing the same SRC, DST, SPT and DPT values from the ORIG part of the message belong to the same connection, unless they are not from the same host.

parser p_correlate_session_data {
        where(match("ORIG" value("MESSAGE")))
        trigger(match("DESTROY" value("MESSAGE")))
        having( "${UNIXTIME}@2" ne "1" )
            value("event.start" "${ISODATE}@2")
            value("event.end" "${ISODATE}@1")
            value("event.duration", "$(- ${UNIXTIME}@1 ${UNIXTIME}@2)")
            value("MESSAGE" "Session completed; client='${outbound.source.ip}'; server='${outbound.destination.ip}'; destination_port='${outbound.destination.port}; protocol='${network.transport}'; session_lenght='${event.duration}'\n")
        # destroy events sometimes arrive later than 2 minutes, even when a client app is already closed (ssh, telnet)

The context will be closed and evaluated either when a message arrives which matches the filter specified in having() or the timeout() occurs.
Important! The timeout is currently set to 10 minutes. Connections longer than 10 minutes will set to 10 minutes in Elasticsearch.

The evaluation will aggregate the context and creates new name-value pairs specified with values(). For example, it creates a new MESSAGE. This message is logged the same place where the received logs are stored. This is how such a new message looks like.

   "tags" : "openwrt",
   "host" : {
      "name" : "firewall",
      "ip" : ""
   "inbound" : {
      "source" : {
         "port" : "80",
         "bytes" : "44",
         "packets" : "1",
         "ip" : ""
      "destination" : {
         "ip" : "",
         "port" : "36351"
   "message" : "Session completed; client=''; server=''; destination_port='80; protocol='TCP'; session_lenght='3'\n",
   "ecs" : {
      "version" : "1.0.0"
   "@timestamp" : "2019-07-03T14:55:41+02:00",
   "outbound" : {
      "source" : {
         "packets" : "1",
         "port" : "36351",
         "address" : "",
         "bytes" : "40",
         "ip" : ""
      "destination" : {
         "geo" : {
            "region_iso_code" : "EX",
            "location" : "11.000000,12.000000",
            "city_name" : "Smithwille",
            "continent_name" : "Antarctica",
            "country_iso_code" : "AN",
            "region_name" : "",
            "country_name" : ""
         "address" : "",
         "port" : "80",
         "ip" : ""
   "event" : {
      "duration" : "3",
      "end" : "2019-07-03T14:55:41+02:00",
      "start" : "2019-07-03T14:55:38+02:00"
   "network" : {
      "transport" : "TCP"

Actually I only need the parsed name-value pairs for Elasticsearch. The existence of this message indicates that everything is available. Therefore I will filter on this message later.

Add reverse DNS data to OpenWRT messages with syslog-ng’s Python parser

Although getting reverse DNS can be questionable I still find it useful. You just need to keep in mind, the domain name given back by reverse DNS queries does not necessarily correspond to the domain name of a server you visited. What you usually have is the domain name of an edge router or reverse proxy of CDN networks your OpenWRT connected to.

Unfortunately syslog-ng does not support running name resolution on any macro. However we can write such a parser in Python. By using such a parser I managed to resolve host names for clients and servers and add them to the session metadata.

There is an existing Python code to do reverse DNS resolution, but it needs some changes to work on any macros. Because of the current license of that post does not explicitly permit changes and redistribution of the content. I only provide a patch you may need to apply on top of that. (I am in discussion with the owner to get proper permissions and update this post accordingly.)

>     def init(self, options):
>         self.ip = options["ip"]
>         self.result_key = options["result"]
>         return True
<         ipaddr_b = log_message['suricata.dest_ip']
>         ipaddr_b = log_message[self.ip]
<             log_message['parsed.dest.hostname'] = hostname
>             log_message[self.result_key] = hostname

Specifying the python parsers should come after the previous parsers. You can do it like this.

parser p_reversedns_server {
            "ip" "outbound.destination.ip"
            "result" "outbound.destination.address"

My OpenWRT server assigns domain names for clients with fixed IP addresses. Therefore I do reverse lookup on client IP addresses too. If you would like to do the same, then create another parser but with options pointing to nf_orig.SRC and hostname.client. Again for the complete configuration check the GitHub repository.

Sending conntrack sessions from OpenWRT to Elasticsearch

Sending network traffic data of your home network from OpenWRT to syslog-ng is a great thing. But what is more cooler is to send it to Elasticsearch and create visualizations and reports. You can create nice visualizations once you have proper data type mapping set. Regardless what software you use for log transfer data type mapping must be in advance.

Creating data type mappings for connection session details

Explicit mapping should be created manually by using Elasticsearch’s PUT API. You can do this in Dev Tools → Console. If this is new to you, then you should check my previous step by step guide about it.
Because of the length of the mapping file I provide a downloadable version in the following file on GitHub.

Please note that this mapping also specifies a template for the index names it will match on. In my case it is network-*.

Monitoring my home network with examples

I already created videos about how you can make different type of visualizations in Kibana. You should definitely check the videos if you are stuck. Therefore this time I picked up two recent cases because I wanted to focus on what benefits I could get by having network data in Elastic.

Where and how are my images uploaded by using a web printing service?

I wanted to print dozens of family photos on paper. I decided to use CEWE’s (Rossmann) service to do that. They even have a Qt based software for Linux to place orders.

I wanted to feel secure as I am about to upload private data to someone else’s computer. Where are they uploaded? Are those services use HTTPS for transferring my precious family photos? Let’s find that out.

Note: My actions did take place between 21:00 and 22:30 on March 17th, 2019.

  • In the browser’s address bar I noticed that registration and uploading takes place on “photoprintit” domains via HTTPS. Let’s check the traffic belonging to those domains. Using the following regex query outbound.destination.address:*photoprint* on the dashboard gave me the following results.Kibana Dashboard showing traffic syslog from photoprintit domain via OpenWRT
  • On the top left corner in Coordinate Map visualization we can see that all traffic goes to Germany (I am located in Hungary). As a result I checked the service’s Data Privacy document which also shows that both their web site and the hosting is located in Germany. Kibana Geo Map photoprintit trafficI am not surprised as Rossmann is a German company, however it is quite funny that I had to send my photos to Germany for printing so they will deliver them back to Hungary.
  • Do they use HTTPS for the traffic? Let’s use a Data Table visualization to see that. It seems like that there are some traffic on 80, however most of the traffic goes to 443 (HTTPS).Kibana Data Table photoprintit traffic details

I did not have to do any extra work to get these information because GeoIP, Reverse DNS, traffic size and details are all there. As a result I could do the same exercise with any other web site.

Is there any device on my network phoning home?

Is there any network traffic going to a specified country, for instance is there anything going to China unattended? (I could choose any other country.)

Note: In the following examples I excluded a client host where I made explicit connections to Chinese websites. Data is from 11 day long period.

  • We can filter on a country by using outbound.destination.geo.country_name:China.Kibana Dashboard China details
  • According to the results there are two devices which communicate to that country. The device tplink.lan is a WiFi Access Point running on OpenWRT. The other device Galaxy-A5-2016.lan is a smartphone connecting to that access point over WiFi.
    Let’ see the details on a different Data Table visualization again.Kibana Data Table visualization China traffic details

Apparently there are communication going outwards of my home network, however I still do not know what is inside of them. That WiFi Access Point used NAT a couple of days ago until I disabled it, seems like there may be duplicate entries.

A logical next step would be to run tcpdump on OpenWRT parameterized to only capture and save TCP packets going on to those IP addresses. You can further analyze the packet captures  with other tools. But this is a topic of another post I will write later.


I hope that with the help of this blog post anyone can monitor their home network with OpenWRT and syslog-ng. However it is not necessary to use Elasticsearch. You can use any other tools you like.

If this post was helpful for you, then please share it. Even more should you have anything to ask then feel free to make any comments below. I will highly appreciate it.

58 thoughts on “Monitor home network traffic with OpenWRT and Syslog-ng

  1. Gandalf


    Great tutorial…

    I have installed SELKS 5, Debian based, and upgraded syslog-ng to the latest from the unnoficial repository, to get the python parser support.
    Still no luck and syslog-ng fail with an error,

    [email protected]:/etc/syslog-ng$ sudo syslog-ng -Fedv
    [2019-07-01T09:22:17.096428] Error loading Python module; module='_syslogng_main', exception='exceptions.ImportError: No module named _syslogng_main'
    ImportError: No module named _syslogng_main
    [2019-07-01T09:22:17.096736] Error looking Python parser class; parser='p_reversedns_server', class='SngResolver', exception='None'
    [2019-07-01T09:22:17.096939] Error printing proper Python traceback for the exception, printing the error caused by print_exception() itself;
    SystemError: NULL object passed to Py_BuildValue
    [2019-07-01T09:22:17.097286] Error initializing message pipeline; plugin_name='python', location='/etc/syslog-ng/conf.d/network.conf:36:5'

    I have taken the config files from the github…
    Can you help about this ?

    Have you plans about simplifying the monitoring tutorial from OpenWRT with ELK (or equivalent)…


    1. Balázs Németh Post author


      I remember in 3.22 the developers incorporated some changes regarding python module support.

      However my code still works. I am going to update the github repo with the latest changes I have. Give it a try and let me know if still breaks.

      You shall also compare the available plugins in your installation with mine.

      # syslog-ng -V
      syslog-ng 3 (3.22.1)
      Config version: 3.22
      Installer-Version: 3.22.1
      Module-Directory: /usr/lib64/syslog-ng
      Module-Path: /usr/lib64/syslog-ng
      Include-Path: /usr/share/syslog-ng/include
      Available-Modules: add-contextual-data,affile,afprog,afsocket,afstomp,afuser,appmodel,basicfuncs,cef,confgen,cryptofuncs,csvparser,date,dbparser,disk-buffer,examples,graphite,hook-commands,json-plugin,kvformat,linux-kmsg-format,map-value-pairs,pseudofile,sdjournal,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,tfgetent,xml,mod-python,geoip-plugin,geoip2-plugin,http
      Enable-Debug: off
      Enable-GProf: off
      Enable-Memtrace: off
      Enable-IPv6: on
      Enable-Spoof-Source: on
      Enable-TCP-Wrapper: on
      Enable-Linux-Caps: on
      Enable-Systemd: on

      ps: I have some plans moving away from the python parser. I know that Logstash can do geoip resolving, however it is way to resource heavy.
      Recently I changed my logging settings and I am getting correct DNS data from dnsmasq running on OpenWRT. The only thing preventing me posting it here is the lack of integration with ulogd2 logs.

  2. Gandalf

    look like better and promising with this version of the python parser…

    Now I still have an issue but it is now in the Kibana Console;
    The PUT _template…… get me stuck in an error :

    "error": {
    "root_cause": [
    "type": "mapper_parsing_exception",
    "reason": "Root mapping definition has unsupported parameters: [nf_orig : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}] [nf : {properties={SESSION_END={type=date}, SESSION_START={type=date}, SESSION_LENGTH={type=integer}}}] [geoip2 : {properties={location2={type=geo_point}}}] [nf_reply : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}]"
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [properties]: Root mapping definition has unsupported parameters: [nf_orig : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}] [nf : {properties={SESSION_END={type=date}, SESSION_START={type=date}, SESSION_LENGTH={type=integer}}}] [geoip2 : {properties={location2={type=geo_point}}}] [nf_reply : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}]",
    "caused_by": {
    "type": "mapper_parsing_exception",
    "reason": "Root mapping definition has unsupported parameters: [nf_orig : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}] [nf : {properties={SESSION_END={type=date}, SESSION_START={type=date}, SESSION_LENGTH={type=integer}}}] [geoip2 : {properties={location2={type=geo_point}}}] [nf_reply : {properties={BYTES={type=integer}, DST={type=ip}, SRC={type=ip}, SPT={type=integer}, DPT={type=integer}, PKTS={type=integer}}}]"
    "status": 400

    Hope you’ll help me on this also, tanks you…

    PS : will try to get test on your next release, do you preffer issue on your github ? or comment here is ok ?

    1. Balázs Németh Post author


      I think the problem is with the syntax. The mapping syntax changed in Elastic 7.x. The one uploaded to GitHub is for v7.x
      If you are using and older version then try to add back the the nested type directive. In my case it was called ‘test’. Check the diff for hints.

      I plan to rework the current mapping as it is totally ad-hoc and does not follow ECS. I try to do it this week but it is not a promise. 🙂

      Commenting here is okay. Once the visibility of the github repos may arise I will switch over there.

  3. Gandalf

    Just upgraded from 6.8.1 to 7.2.0 with success…

    I get now
    “acknowledged” : true

    But still get no indexes network-* nor any data…
    May be I still misunderstood how kibana and elastcisearch do work…

    1. Balázs Németh Post author

      The basic procedure is this:

      1) upload index mapping which expects the index names matching “network-*”. As I understand you already did that.
      2) Send data into Elastic with syslog-ng
      3) Create index pattern in Kibana matching index names “network-*”, then go to Discover. You should be able to see the documents there.

  4. Gandalf

    I have to verify the hostname in the syslog-ng/network.conf like the ip from the /var/log/network/IP
    I also put the url to elastic as in syslog-ng/network.conf
    Then I get datas uploaded from my OpenWRT (ulogd > syslog-ng > elasticsearch)
    I have created the index with network and now I can discover datas…

    Will follow with making dashboards ! (hope to)


  5. Gandalf

    got some errors in syslog-ng like :
    [2019-07-03T09:44:42.267530] Parsing failed, template function's second argument is not a number; function='-', arg2=''

    1. Balázs Németh Post author

      It is hard to determine the cause of this without context. I’ve been reworking the network.conf and the Elastic part since yesterday to make it more easy to use and better comply with ECS.
      If you are interested I will upload the changes to github in the following days and update the blog post accordingly.

        1. Balázs Németh Post author

          I have updated the blog post as well as the GitHub page with the latest changes.
          It is better to use it from scratch. You may need to DELETE _template/network_openwrt before you upload the new one.
          I also exported my visualizations and dashboards that you can import in Kibana -> Saved objects.

          Let me know how it works for you.

          1. Balázs Németh Post author

            Thank you very much.

            I am happy that what I’ve done could be useful for others too.

  6. Gandalf

    got also theses errors loged ;
    [2019-07-03T09:49:17.774850] Incoming log entry from journal; message='{"type":"log","@timestamp":"2019-07-03T07:48:59Z","tags":["error","task_manager"],"pid":373,"message":"Failed to poll for work: [cluster_block_exception] index [.kibana_task_manager] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]; :: {\"path\":\"/.kibana_task_manager/_update/Maps-maps_telemetry\",\"query\":{\"if_seq_no\":91,\"if_primary_term\":13,\"refresh\":\"true\"},\"body\":\"{\\\"doc\\\":{\\\"type\\\":\\\"task\\\",\\\"task\\\":{\\\"taskType\\\":\\\"maps_telemetry\\\",\\\"state\\\":\\\"{\\\\\\\"runs\\\\\\\":1,\\\\\\\"stats\\\\\\\":{\\\\\\\"mapsTotalCount\\\\\\\":0,\\\\\\\"timeCaptured\\\\\\\":\\\\\\\"2019-07-02T09:58:11.782Z\\\\\\\",\\\\\\\"attributesPerMap\\\\\\\":{\\\\\\\"dataSourcesCount\\\\\\\":{\\\\\\\"min\\\\\\\":0,\\\\\\\"max\\\\\\\":0,\\\\\\\"avg\\\\\\\":0},\\\\\\\"layersCount\\\\\\\":{\\\\\\\"min\\\\\\\":0,\\\\\\\"max\\\\\\\":0,\\\\\\\"avg\\\\\\\":0},\\\\\\\"layerTypesCount\\\\\\\":{},\\\\\\\"emsVectorLayersCount\\\\\\\":{}}}}\\\",\\\"params\\\":\\\"{}\\\",\\\"attempts\\\":0,\\\"'
    [2019-07-03T09:49:17.774864] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input='{"type":"log","@timestamp":"2019-07-03T07:48:59Z","tags":["error","task_manager"],"pid":373,"message":"Failed to poll for work: [cluster_block_exception] index [.kibana_task_manager] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]; :: {\"path\":\"/.kibana_task_manager/_update/Maps-maps_telemetry\",\"query\":{\"if_seq_no\":91,\"if_primary_term\":13,\"refresh\":\"true\"},\"body\":\"{\\\"doc\\\":{\\\"type\\\":\\\"task\\\",\\\"task\\\":{\\\"taskType\\\":\\\"maps_telemetry\\\",\\\"state\\\":\\\"{\\\\\\\"runs\\\\\\\":1,\\\\\\\"stats\\\\\\\":{\\\\\\\"mapsTotalCount\\\\\\\":0,\\\\\\\"timeCaptured\\\\\\\":\\\\\\\"2019-07-02T09:58:11.782Z\\\\\\\",\\\\\\\"attributesPerMap\\\\\\\":{\\\\\\\"dataSourcesCount\\\\\\\":{\\\\\\\"min\\\\\\\":0,\\\\\\\"max\\\\\\\":0,\\\\\\\"avg\\\\\\\":0},\\\\\\\"layersCount\\\\\\\":{\\\\\\\"min\\\\\\\":0,\\\\\\\"max\\\\\\\":0,\\\\\\\"avg\\\\\\\":0},\\\\\\\"layerTypesCount\\\\\\\":{},\\\\\\\"emsVectorLayersCount\\\\\\\":{}}}}\\\",\\\"params\\\":\\\"{}\\\",\\\"attempts\\\":0,\\\"scheduledAt\\\":\\\"2019-07-02T09:58:11.518Z\\\",\\\"runAt\\\":\\\"2019-07-03T07:49:59.504Z\\\",\\\"status\\\":\\\"running\\\"},\\\"kibana\\\":{\\\"uuid\\\":\\\"21ca7904-ac5c-4314-b75e-3996fd5c3c42\\\",\\\"version\\\":7020099,\\\"apiVersion\\\":1}}}\",\"statusCode\":403,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"cluster_block_exception\\\",\\\"reason\\\":\\\"index [.kibana_task_manager] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];\\\"}],\\\"type\\\":\\\"cluster_block_exception\\\",\\\"reason\\\":\\\"index [.kibana_task_manager] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];\\\"},\\\"status\\\":403}\"}"}', marker='@cim:'

    1. Balázs Németh Post author

      This looks to be unrelated to the topic. Seems like there is a permission issue in Kibana.

      1. Gandalf

        yes, looks like my upgrade of SELKS do broke some access…

        Will try to do a fresh install on a new Debian of ELK…
        If you have advice about howto and tutorial, thanks… 😉

        1. Gandalf

          Look okay with the latest from your github…

          I used a new fresh SELKS install, upgrade syslog-ng to latest from unnofficial repository, upgraded ElasticSearch, LogStash and Kibana to 7.2 from official repository.
          I also upgraded the latest java-open.

          Then I downloaded the GEOIP with your script from the fail2ban post.
          Configure syslog-ng and add network-* index from kibana (after a small data added).
          I also added your templates and dashboards (fine, thank you)…

          thanks again…

          a samll tips, to help; I have done this on network.conf that make the configuration more “generic/portable”
          [email protected]:~# cat /etc/syslog-ng/conf.d/network.conf
          @define network_ip “”
          @define router_hostname “netgear”
          @define elastic_host “”

  7. Gandalf

    A new question rise to me.
    Is it possible to log any traffic with ulogd, like on a switch listening mode ?
    Not the traffic which get through the routeur, but the traffic listened on a switch ?

    It’s a little off topic about your tutorial, but it may quickly be adapted to get a mirror port device passively listen the home network traffic without requesting to get a OpenWrt Routeur, but a sepcific device for network traffic analysis…

    Thanks if you can help me to understand how to make this …

    1. Balázs Németh Post author

      Depending on what log a given switch could produce, you can integrate them into Elasticsearch too.
      If if can run ulogd then it should be easy.
      If it is NetFlow then that should be also achievable.
      I picked my main router as it has the singe point of view about connections.

      OpenWRT can be also used to produce netflow data. Also it can mirror the traffic via iptables to other host where you can analyze it with Suricata. I have plans to do it, but ulogd seemed to be a low hanging fruit compared to the complexity and network requirements of port mirroring.

      1. Gandalf

        Héllo again…

        I am still testing ulogd with a bridge interface… All may works but I am not getting like tcpdump from ulogd, only UDP but not the full TCP ? Can you help about what I am missing ?

        1. Balázs Németh Post author

          Hello again,

          Do you mean that ulogd is set up to send logs to remote syslog?
          Or you only get logs about UDP packets but not TCP?

          1. Balázs Németh Post author

            First please compare your ulogd config to mine.
            I would also check TCP traffic with tcpdump to see whether are there any TCP traffic at all.
            Try enabling promiscuous mode on your interfaces to see all the traffic it could possibly see.
            If the issue is not with ulogd (I doubt it) then probably it is in your network.

          2. Gandalf

            Thanks for the tips;
            I have checked again your configuration (backuped mine and set yours) and same behaviour of ulogd…
            I have already tcpdump and all is okay with it… full paquet logging when I specify my bridged lans network interface…
            ulogd receive only paquets from the WAN interface, which is used for ssh and syslog/ulog, and also http (for luci interface)…

            My bridged interface, without IP stack do not look to logging into ulog…
            I have tested with IP stack, same behaiour…
            I also made promisc mode on all networks interfaces, but get nothing more…

            Can I use ulogd to specifically log an interface ? (my bridged one)…
            Can I use ulogd with the NFCT plugin to log ethernet bridged interface (like a TAP passive monitoring interface) ?

            If I disable or stop iptables, i get nothing at all, ulogd become full silent !
            Did I need some IPTABLES rules to get my bridge interface talking with ulogd ?

            Thanks at all for any help you can give me…

          3. Balázs Németh Post author

            On my OpenWRT router I also have a bridge called br-lan and I did not had to configure anything related to interfaces. My WAN interface is not a member of a bridge.
            However as far as I know NFCT (libnetfilter_conntrack) only reads data from the kernel’s connection track table. It is not interface specific from this perspective.
            Try to install the ‘conntrack’ package in OpenWRT to see what do you have in the conntrack table.
            As long as you cannot see there what you want to see then ulogd would not see either.

          4. Balázs Németh Post author

            I hope you will get help there.
            You are welcome.

          5. Gandalf

            Hy again for some feedback…

            I am thinking now that conntrack will be useless on a switch bridged monitoring solution…
            Because there is no connection to track…

            I have get somme success with the TCPDUMP, or while mirroring the traffic…
            But only with NFLOG for logging purpose…!

            Is your solution is adaptable to NFLOG instead of NFCT ?


          6. Balázs Németh Post author


            The syslog-ng’s configuration was especially made for NFCT support. I assume NFLOG would provide logs in the same good old iptables log format.
            If that is the case then I would not bother with syslog-ng to process and send it to Elastic, but only store the logs somewhere and use Filebeat’s iptables module to send them directly to Elasticsearch.


          7. Gandalf

            Okay, thanks for the advice…

            But I really like the remote OpenWRT monitoring sender to a PC/Server analysis systems like…

            May be I will find such a tutorial somewhere on Internet…


          8. Balázs Németh Post author

            What I suggested will do the same. You can use NFLOG as I assume its output will be syslog messages on OpenWRT. What you can send to a remote syslog host the same way as with NFCT with syslog-ng (client side).
            I just would not want to do the processing and transformation of logs with syslog-ng (server side) but use Filebeat to read up the files stored by syslog-ng and transfer them to Elasticsearch.


          9. Gandalf

            will try this…

            But looks like all is working now !…
            conntracking from my bridge to the remote server SELKS…

            Great, and thanks for the help, advice and sharing !

          10. Balázs Németh Post author

            Glad it works now. Do you know what was the issue?
            You’re welcome.

          11. Gandalf

            The issue is about the DNS resolve…

            The syslog-ng of my server looks like to flood the DNS and make so lot of UDP 54 call that the conntrack is overhead !

            I have to tweak it now to get syslog-ng call not flood my DNS…

            Any advice ? (again lol ;-))

          12. Balázs Németh Post author

            I know what you are talking about. I have experienced the same when I checked the collected metrics. DNS traffic was responsible for more than 60% of my traffic.
            I tried to set up a local dnsmasq to cache requests but because of low TTLs it did not really had the effect I was expecting.
            Some months ago I decided to turn it off and looking for other options.
            Currently I have only the following ideas.

            1. Rewrite the reverse DNS code in the python parser to maintain a cache of resolved hosts for longer period of time than DNS TTL. Although it have a risk of being outdated, it will reduce DNS traffic considerably.
            2. Also you could set up a syslog-ng filter to dismiss traffic logs of normal and reverse DNS calls of syslog-ng itself.
            3. Set up syslog-ng to resolve DNS names locally. You can look for the most frequent domains and cache them in /etc/hosts. Of course this could be expensive from maintenance point of view unless you automate it. Hint: elasticsearch can give you pretty easily the top N number of domain names. This could be a low-hanging fruit.
          13. Balázs Németh Post author

            That DNS filter is for Logstash which is a heavy player regarding its system resource demands.
            Of course you could do that. I simply did not want to have Logstash as I wanted a lightweight solution and Elasticsearch itself is already pretty heavy for my HP Microserver.

  8. Oupsman

    Great tutorial, I’m struggling with it and almost got something working.

    I’m quite surprised to see that long downloads aren’t reported and surprised by this line :


    Why correlate using the outbound source ip twice ? Shouldn’t we read this :



    1. Balázs Németh Post author

      Thank you.

      It was a typo I missed to catch for so long time. I have just fixed it both in the blog and in GitHub.
      Indeed we would need the ‘outbound.source.port’ instead of ‘outbound.source.ip’.

      There is a timeout set to 600 seconds to wait for context closing events ([DESTROY] logs). This is not ideal but prevents high memory usage on busy servers. When this timeout is reached the context is closed and its duration is effectively ‘faked’ to 10 minutes. You can increase it if you like.

      Thanks again for letting me know the mistake I made.
      In cases where let say a download takes a lot of time

  9. Chris

    Having a weird issue where ulodg is showing up in remote syslog messages, but no data is being logged to elasticsearch

    I believe I followed everything correctly, only difference is I run elasticsearch on

  10. Chris

    I figured it out, my router wasnt sending hostname over syslog it was sending its ip address.

    I set @define router_hostname to its ip address and I have data.

  11. Tony

    I’d love to get this working, but I’m stuck at the parsing.
    The ulogs aren’t csvparsed at al, so the rest isn’t working either.
    This is the log that comes in:
    Nov 8 13:54:50 A.B.C.254 ulogd[12099]: [NEW] ORIG: SRC=A.B.C.1 DST=A.B.C.254 PROTO=UDP SPT=57024 DPT=53 PKTS=0 BYTES=0 , REPLY: SRC=A.B.C.254 DST=A.B.C.1 PROTO=UDP SPT=53 DPT=57024 PKTS=0 BYTES=0
    Nov 8 13:54:50 A.B.C.254 ulogd[12099]: [DESTROY] ORIG: SRC=A.B.C.1 DST=A.B.C.254 PROTO=UDP SPT=47344 DPT=53 PKTS=1 BYTES=70 , REPLY: SRC=A.B.C.254 DST=A.B.C.1 PROTO=UDP SPT=53 DPT=47344 PKTS=1 BYTES=70

    If I’m correct, when only applying the csvparsing, the output should be:
    Nov 8 13:54:50 A.B.C.254 ulogd[12099]: [NEW] ORIG: SRC=A.B.C.1 DST=A.B.C.254 PROTO=UDP SPT=57024 DPT=53 PKTS=0 BYTES=0
    Nov 8 13:54:50 A.B.C.254 ulogd[12099]: [DESTROY] ORIG: SRC=A.B.C.1 DST=A.B.C.254 PROTO=UDP SPT=47344 DPT=53 PKTS=1 BYTES=70

    This is what’s in the conf file:

    source s_network{
    tcp(ip(A.B.C.1) port(514));
    parser p_csv {
    csv-parser(columns("ORIG", "REPLY")
    destination d_network{

    log {

    How can I trace the parsing?

  12. Balázs Németh Post author


    Please note that you may want to replace tcp() driver to network() as tcp() is obsolate. (
    However this may not change what you experience.

    The parser will be used on ${MESSAGE} part of the message. ie, everything after “ulogd[1234]:”.
    Therefore the macros ORIG and REPLY after parsed by csv-parser should look similar to this.

    ORIG => "[NEW] ORIG: SRC=A.B.C.1 DST=A.B.C.254 PROTO=UDP SPT=57024 DPT=53 PKTS=0 BYTES=0"

    Maybe the namings are confusing a bit but you see where it is going.

    I quickly checked the sample of logs you provided with the one I have and they look identical regarding their format so the parser config should be fine. What I can think of is that the log message does not reach the parser at all.

    The troubleshooting approach you made is a good one. Remove everything and enable further actions one-by-one.
    The debug, verbose, trace modes can be enabled by syslog-ng-ctl to troubleshoot syslog-ng issues. As far as I recall trace is required to see how filters are evaluated.
    Unfortunately sometimes all of them must be turned on and it will be very noisy. Try to keep the volume of your logs low while these settings are on.

    Good luck.

  13. David

    This is great. Is it possible to offer the Kibana / Server side of things as a Docker container or compose file with the configs?


    1. Balázs Németh Post author

      Hi David,

      What do you mean on Kibana part? Sure you can run the whole Elastic stack in Docker, so as the syslog-ng server. The only ‘dockerless’ part is ulogd on OpenWRT.


  14. Bobby

    Thank you for the elaborate guide and ready to use files. I have completed most of the steps but am stuck with an error during creating mappings from the Kibana webgui. Could you please help me with this

    Unable to create template
    [mapper_parsing_exception] A dynamic template must be defined with a name

    I am a total newbie to elasticsearch and kibana. I am using Elasticsearch 7.8.0 and Kibana 7.8.0

    1. Balázs Németh Post author

      Hi Bobby,

      Try to change the PUT request to look like this.

      PUT _template/network_openwrt?include_type_name

      I’ve changed a lot about these things recently to be able to use the SIEM module, but have not published yet. I’m going to find some time to update this post.
      I hope it works for you.

  15. Bobby


    Thank you for the prompt response. I am not sure how to incorporate that change.

    I can see the final text that goes to elasticsearch to create the template, but there is no way to modify it. I will see if there is another interface from which i can do it

    Maybe I will wait for your updated posts on the topic. Thanks again !!

  16. Bobby


    I found the Kibana console and put the content there. Now I am getting the following error

    ———————————Error on kibana console———————————-
    “error” : {
    “root_cause” : [
    “type” : “mapper_parsing_exception”,
    “reason” : “Root mapping definition has unsupported parameters: [inbound : {properties={destination={properties={address={type=keyword}, port={type=long}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [outbound : {properties={destination={properties={address={type=keyword}, port={type=long}, geo.location={type=geo_point}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [host : {properties={hostname={type=keyword}, ip={type=ip}, name={type=keyword}, type={type=keyword}, architecture={type=keyword}}}] [event : {properties={duration={type=long}, start={type=date}, end={type=date}}}]”
    “type” : “mapper_parsing_exception”,
    “reason” : “Failed to parse mapping [properties]: Root mapping definition has unsupported parameters: [inbound : {properties={destination={properties={address={type=keyword}, port={type=long}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [outbound : {properties={destination={properties={address={type=keyword}, port={type=long}, geo.location={type=geo_point}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [host : {properties={hostname={type=keyword}, ip={type=ip}, name={type=keyword}, type={type=keyword}, architecture={type=keyword}}}] [event : {properties={duration={type=long}, start={type=date}, end={type=date}}}]”,
    “caused_by” : {
    “type” : “mapper_parsing_exception”,
    “reason” : “Root mapping definition has unsupported parameters: [inbound : {properties={destination={properties={address={type=keyword}, port={type=long}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [outbound : {properties={destination={properties={address={type=keyword}, port={type=long}, geo.location={type=geo_point}, ip={type=ip}}}, source={properties={address={type=keyword}, port={type=long}, bytes={type=long}, ip={type=ip}, packets={type=long}}}}}] [host : {properties={hostname={type=keyword}, ip={type=ip}, name={type=keyword}, type={type=keyword}, architecture={type=keyword}}}] [event : {properties={duration={type=long}, start={type=date}, end={type=date}}}]”
    “status” : 400
    —————————————Error on kibana console————————————–
    This looks like the same as Gandalf posted on 07/01/2019

    I have Elasticsearch 7.8.0 and Kibana 7.8.0 . I sincerely hope i am not being a nuisance


    1. Balázs Németh Post author


      I rather try to post my updates soon than figuring out why the version I have is broken.


  17. User

    Well i have some huge problems with active connections in my OpenWRT router.
    My router send logs via TCP port 601 to remote syslog-ng running in docker container.
    Elastic show’s data but after while active connections at my router site shows 99% allowed connections (about 16k udp connections from my syslog-ng) and OpenWRT router start to DROP connections.

  18. n

    great tutorial, thanks for your work!
    just one thing: I was looking at the makefile for ulogd-mods-extra and it looks like that pulling that package actually installs the whole suite, but (unless I missed something) most of those mods are unused for this tutorial


Leave a Reply

Your email address will not be published. Required fields are marked *

81 − seventy two =