All Apps and Add-ons

Unable to override host value using REGEX

att35
Builder

Hi,

We have Linux Audit data coming from OSSEC which is getting correctly parsed by Splunk Add-on for Audit. One field which still gets the default value is "host". Instead of extracting the value of the original reporting host, it is set to the OSSEC server which is sending the data.

To fix this, I added the same extraction used by OSSEC App, but this works only for a new extracted field (using REPORT), but not when overriding the host itself.

e.g. The following works correctly. It extracts and creates a new field called "reporting_host".

transforms.conf

[linux-audit-host]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = reporting_host::$1

props.conf

REPORT-host1 = linux-audit-host

But same REGEX, when used to override "host" makes no change. host value goes back to default.

transforms.conf

[hostoverride]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf

TRANSFORMS-host = hostoverride

Here is sample of the RAW data which is parsed using the above REGEX

2016 Jun 07 16:13:26 (Hostname) any->/var/log/audit/audit.log

Since the field "reporting_host" is parsed correctly, I am assuming the REGEX works. Not sure why the host override fails.

Kindly Assist.

Thanks,

~ Abhi

0 Karma
1 Solution

att35
Builder

Both transforms and props.conf were changed on the Indexer, but the mistake was that TRANSFORMS attribute got added under [linux:audit] instead of [ossec_alerts], which is the sourcetype for the original Data. After making the change, we are now Indexing Auditd data correctly. Dashboards on Linux Auditd App populated as well.

Working Config:
transforms.conf

[hostoverride]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf

[ossec_alerts]
TRANSFORMS-ossec_auditd = linux_audit
TRANSFORMS-host = hostoverride

Many Thanks,

~ Abhi

View solution in original post

att35
Builder

Both transforms and props.conf were changed on the Indexer, but the mistake was that TRANSFORMS attribute got added under [linux:audit] instead of [ossec_alerts], which is the sourcetype for the original Data. After making the change, we are now Indexing Auditd data correctly. Dashboards on Linux Auditd App populated as well.

Working Config:
transforms.conf

[hostoverride]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf

[ossec_alerts]
TRANSFORMS-ossec_auditd = linux_audit
TRANSFORMS-host = hostoverride

Many Thanks,

~ Abhi

pkarpushin
Path Finder

I got same issue, but instead of sourcetype value in stanza I use source value.

My config:
props.conf

[source::\\sl-netmon\logs\*.txt]
TRANSFORMS-forcechange_host = kiwi-syslog-preamble-forcechange_host

transforms.conf
[kiwi-syslog-preamble-forcechange_host]

REGEX = ^(?(?\d+-\d\d-\d\d)\s+(?\d\d:\d\d:\d\d)\s+(?\S+.\S+)\s+(?(?:\d+.){3}\d+))?\s+(?.+)

FORMAT = host::$1
WRITE_META = true
REGEX = ^(?:(?:\d+-\d\d-\d\d)\s+(?:\d\d:\d\d:\d\d)\s+(?:\S+.\S+)\s+((?:\d+.){3}\d+))?\s+(?:.+)

This is what I get as a result:
host = Splunk-gtw host = 192.168.102.110 source = \sl-netmon\logs\Access.txt sourcetype = TEST_syslog

Host field doubled in search result for all related events.

0 Karma

monteiroh
Explorer

did u solve the duplicate values?

0 Karma

woodcock
Esteemed Legend

Deploy your configurations (transforms.conf & props.conf) to your Indexers (and Heavy Forwarders), restart all Splunk instances there and then check the events that arrive AFTER the restart (older events will stay "broken").

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The Search time field extraction (REPORT-...) needs to be available where Search is initiated, that's Search Head. I believe you created those changes on SH and it's working as it should. The Index time field extraction (TRANSFORMS-..) should be available on the node doing the data parsing, which is generally Indexer OR heavy forwarder. Did you create those TRANSFORMS- configurations on Indexer/Heavy Forwarder?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...