Auditbeat

[strf0x1]

Hey all. Working with Auditbeat this week to understand how viable to would be to get into SO. I noticed there are some ingest node pipelines for auditd data (via filebeat), but nothing in the Logstash workflow for Auditbeat itself. Is that generally the preferred method for ingesting auditd data today? Thanks for any suggestions.

[wdpless]

This is very interesting. I hope you keep this thread updated as you progress.

[strf0x1]

I tested the workflow of traditional auditd logs through the pipeline and was not successful in getting the logs parsed. They did show up, but only with the raw message field and regular metadata. Decided to just get Auditbeat on something that can reach ElasticSearch, and push some logs directly so that Auditbeat will install it's fancy ingest node pipelines etc. So next step might be to manually edit the logstash pipelines and point output at the new ingest node pipeline. Will report back when I figure that piece out.

[strf0x1]

Alright got some free time today and hacked together a solution for the time being. Would love some feedback from anybody if you think there is a better way to do it.

First install auditbeat on a linux node that can reach the elasticsearch endpoint at port 9200. Configure auditbeat to output to ElasticSearch (and if you want dashboards, also configure Kibana in a similar manner):

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["your_elasticsearch_ip:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"
  ssl.verification_mode: none

Start Auditbeat up, and this will create index templates, ILM, alias etc. in ElasticSearch. You can confirm this worked by going to Stack Management -> Index Lifecycle Policies. There will be a new ILM called auditbeat.

Now stop auditbeat, and comment out the output.elasticsearch section. Uncomment the logstash section like so:

output.logstash:
  # The Logstash hosts
  hosts: ["security_onion_logstash_ip:5044"]

at the very bottom of the auditbeat.yml config, change processors to add a tag:

processors:
  - add_tags:
      tags: [auditbeat]

This will tag all the data coming in to the logstash pipeline, and we can use it so send through a custom output on the search node. Edit /opt/so/saltstack/default/salt/logstash/pipelines/config/so/9500_output_beats.conf.jinja:

{%- if grains['role'] == 'so-eval' -%}
{%- set ES = salt['pillar.get']('manager:mainip', '') -%}
{%- else %}
{%- set ES = salt['pillar.get']('elasticsearch:mainip', '') -%}
{%- endif %}
{%- set ES_USER = salt['pillar.get']('elasticsearch:auth:users:so_elastic_user:user', '') %}
{%- set ES_PASS = salt['pillar.get']('elasticsearch:auth:users:so_elastic_user:pass', '') %}
output {
  if "beat-ext" in [tags] and "import" not in [tags] and "auditbeat" not in [tags] {
    elasticsearch {
      pipeline => "beats.common"
      hosts => "{{ ES }}"
{% if salt['pillar.get']('elasticsearch:auth:enabled') is sameas true %}
      user => "{{ ES_USER }}"
      password => "{{ ES_PASS }}"
{% endif %}
      index => "so-beats"
      template_name => "so-beats"
      template => "/templates/so-beats-template.json"
      template_overwrite => true
      ssl => true
      ssl_certificate_verification => false
    }
  if "auditbeat" in [tags] {
       elasticsearch {
              hosts => "{{ ES }}"
              ilm_rollover_alias => "auditbeat-7.13.4"
              ilm_policy => "auditbeat"
              ssl => true
              ssl_certificate_verification => false
       }
  }
}

Notice the first condition has a new condition (and "auditbeat" not in [tags]), and at the bottom condition, the ILM rollover alias has to match your version of auditbeat. There's probably a better way to define that but haven't figured it out yet.

Then so-logstash-restart for the config to be applied, and you should be good to go. Hope that helps somebody. Probably not production ready way to do it, but I needed this for a small implementation this week.