topic Re: overriding default host field extraction won't work in Getting Data In

overriding default host field extraction won't work

sansay — Sun, 04 Jun 2017 17:40:13 GMT

After spending 4 hours trying to make this work I am out of ideas.
Here is my final setup and it is what I think should work:

Sample event:

Jun  4 09:58:04 10.237.94.1 1 2017-06-04T16:58:03.876Z lvs.fwc.p1087-34 RT_FLOW - RT_FLOW_SESSION_DENY [junos@2636.1.1.1.2.28 source-address="...

props.conf stanza:

[juniper:junos:firewall]
TRANSFORMS-<juniper-fw-host> = <juniper-firewall-host-override>

transforms.conf stanza

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

I tested the REGEX with REX in a search like this:

index=sec_fw sourcetype=juniper:junos:firewall | rex "\s(?P<host2>\w+\.\w+\.\w\d+-*\d*)\s"

And it works, the correct field/value pair appears in the field pane.
My indexing time field extraction should replace the syslog host name with "lvs.fwc.p1087-34".
I read and read the RTFM, followed the instructions to the letter, and yet, it just won't work.
Any help will be very much appreciated.

Re: overriding default host field extraction won't work

rphillips_splk — Sun, 04 Jun 2017 18:30:40 GMT

@sansay

Two things to check.

That your props/transforms are configured on the indexer since this is an index-time field extraction
Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

Re: overriding default host field extraction won't work

sansay — Sun, 04 Jun 2017 20:42:42 GMT

I must be angle-brackets blind.
I have looked at the stanzas many many time over the last 4 hours, and never noticed them.
Thank you so much.