Override host field with event data

[raduand]

Hello,

I am indexing some data from a file monitor and i want to override the host field with data that lays inside the events. Below is a sample of the data and the values i want for the host field with bold.

Mon Oct 09 2017 15:24:18 SE-001 sshd[5905]: Failed password for invalid user postgres from 49.212.64.138 port 4856 ssh2
Mon Oct 09 2017 15:24:13 ACME-005 sshd[2792]: Failed password for nsharpe from 10.2.10.163 port 1148 ssh2
Mon Oct 09 2017 15:24:12 ops-sys-006 sshd[4105]: Failed password for sync from 233.77.49.94 port 4595 ssh2
Mon Oct 09 2017 15:24:19 PROD-MFS-001 sshd[74897]: pam_unix(sshd:session): session closed for user nsharpe by (uid=0)
Mon Oct 09 2017 15:24:07 PROD-MFS-001 su: pam_unix(su:session): session closed for user root

The data is indexed under linux_secure sourcetype. In order to achieve the host overriding, i added one props.conf and one transforms.conf stanza in /etc/system/local on the indexers:

props.conf
[linux_secure]
TRANSFORMS-sethost = set_hostname_linux_secure
SHOULD_LINEMERGE = false

transforms.conf
[set_hostname_linux_secure]
REGEX = (?<=:\d{2}\s).*?(?=\s)
FORMAT = host::$1
DEST_KEY = MetaData:Host

The above configuration is not working, and the events are still indexing with host = the name of the forwarder where they come from.

Any idea what's wrong with this configuration and how can i implement the host overriding?

Thanks a lot!

[lfedak_splunk]

Hey @raduand, if they solved your problem, remember to "√Accept" an answer to award karma points 🙂

[raduand]

problem is not solved yet 🙂

[cpetterborg · Accepted solution]

To me it looks like your problem is in your config. You are calling out $1 in the FORMAT line, but you don't actually have a capture group that you can use. Try this:

REGEX = (?<=:\d{2}\s)(\S+)(?=\s)

It should at least have something in $1 for it to set the host with (the (\S+) will be the only capture group that returns a value).

[raduand]

You are right about the config problem, but even after updating the Regex expression to capture a group that returns a value the host overriding still doesn't work.

Any other suggestion or idea how to troubleshoot this?

Thank you!

[hardikJsheth]

Are you using distributed environment???

These configuration should be added on indexer if you are using universal forwarder.

[raduand]

Yes, i am using distributed environment. The data is coming from a heavy forwarder. The configuration was placed on the indexers and the host overriding was not working.

I just placed props.conf and transforms.conf on the Heavy Forwarder and it's working like a charm.

I need to know why the indexers are not performing this parsing.

[raduand]

I'm running Splunk 6.6.3

[hardikJsheth]

If you are using Heavy Forwarder then you have to place these configurations on HF only. Because with HF, Splunk completes parsing on HF itself and indexer only indexes data. Please refer following link to know more about Splunk indexing.
https://wiki.splunk.com/Community:HowIndexingWorks

[raduand]

Great information, thanks! problem solved then.