Unable to override host value using REGEX

[att35]

Hi,

We have Linux Audit data coming from OSSEC which is getting correctly parsed by Splunk Add-on for Audit. One field which still gets the default value is "host". Instead of extracting the value of the original reporting host, it is set to the OSSEC server which is sending the data.

To fix this, I added the same extraction used by OSSEC App, but this works only for a new extracted field (using REPORT), but not when overriding the host itself.

e.g. The following works correctly. It extracts and creates a new field called "reporting_host".

transforms.conf

[linux-audit-host]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = reporting_host::$1

props.conf

REPORT-host1 = linux-audit-host

But same REGEX, when used to override "host" makes no change. host value goes back to default.

transforms.conf

[hostoverride]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf

TRANSFORMS-host = hostoverride

Here is sample of the RAW data which is parsed using the above REGEX

2016 Jun 07 16:13:26 (Hostname) any->/var/log/audit/audit.log

Since the field "reporting_host" is parsed correctly, I am assuming the REGEX works. Not sure why the host override fails.

Kindly Assist.

Thanks,

~ Abhi

[att35 · Accepted solution]

Both transforms and props.conf were changed on the Indexer, but the mistake was that TRANSFORMS attribute got added under [linux:audit] instead of [ossec_alerts], which is the sourcetype for the original Data. After making the change, we are now Indexing Auditd data correctly. Dashboards on Linux Auditd App populated as well.

Working Config:
transforms.conf

[hostoverride]
REGEX = \d{4} \w+ \d+ [\d:]+ \(([^\)]+)\) (\S+)->
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf

[ossec_alerts]
TRANSFORMS-ossec_auditd = linux_audit
TRANSFORMS-host = hostoverride

Many Thanks,

~ Abhi

[pkarpushin]

I got same issue, but instead of sourcetype value in stanza I use source value.

My config:
props.conf

[source::\\sl-netmon\logs\*.txt]
TRANSFORMS-forcechange_host = kiwi-syslog-preamble-forcechange_host

transforms.conf
[kiwi-syslog-preamble-forcechange_host]

REGEX = ^(?(?\d+-\d\d-\d\d)\s+(?\d\d:\d\d:\d\d)\s+(?\S+.\S+)\s+(?(?:\d+.){3}\d+))?\s+(?.+)

FORMAT = host::$1
WRITE_META = true
REGEX = ^(?:(?:\d+-\d\d-\d\d)\s+(?:\d\d:\d\d:\d\d)\s+(?:\S+.\S+)\s+((?:\d+.){3}\d+))?\s+(?:.+)

This is what I get as a result:
host = Splunk-gtw host = 192.168.102.110 source = \sl-netmon\logs\Access.txt sourcetype = TEST_syslog

Host field doubled in search result for all related events.

[monteiroh]

did u solve the duplicate values?

[woodcock]

Deploy your configurations (transforms.conf & props.conf) to your Indexers (and Heavy Forwarders), restart all Splunk instances there and then check the events that arrive AFTER the restart (older events will stay "broken").

[somesoni2]

The Search time field extraction (REPORT-...) needs to be available where Search is initiated, that's Search Head. I believe you created those changes on SH and it's working as it should. The Index time field extraction (TRANSFORMS-..) should be available on the node doing the data parsing, which is generally Indexer OR heavy forwarder. Did you create those TRANSFORMS- configurations on Indexer/Heavy Forwarder?