Getting Data In

overriding default host field extraction won't work

sansay
Contributor

After spending 4 hours trying to make this work I am out of ideas.
Here is my final setup and it is what I think should work:

Sample event:

Jun  4 09:58:04 10.237.94.1 1 2017-06-04T16:58:03.876Z lvs.fwc.p1087-34 RT_FLOW - RT_FLOW_SESSION_DENY [[email protected] source-address="...

props.conf stanza:

[juniper:junos:firewall]
TRANSFORMS-<juniper-fw-host> = <juniper-firewall-host-override>

transforms.conf stanza

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

I tested the REGEX with REX in a search like this:

index=sec_fw sourcetype=juniper:junos:firewall | rex "\s(?P<host2>\w+\.\w+\.\w\d+-*\d*)\s"

And it works, the correct field/value pair appears in the field pane.
My indexing time field extraction should replace the syslog host name with "lvs.fwc.p1087-34".
I read and read the RTFM, followed the instructions to the letter, and yet, it just won't work.
Any help will be very much appreciated.

1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

sansay
Contributor

I must be angle-brackets blind.
I have looked at the stanzas many many time over the last 4 hours, and never noticed them.
Thank you so much.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Continue Your Federation Journey: Join Session 3 of the Bootcamp Series

To help practitioners build a stronger foundation, we launched the Data Management & Federation ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Casting Call: Compete in Cyber Games

Lights, Camera, SecOps: Apply to Compete in Cyber Games     Think you have what it takes to beat the clock? ...