Getting Data In

overriding default host field extraction won't work

sansay
Contributor

After spending 4 hours trying to make this work I am out of ideas.
Here is my final setup and it is what I think should work:

Sample event:

Jun  4 09:58:04 10.237.94.1 1 2017-06-04T16:58:03.876Z lvs.fwc.p1087-34 RT_FLOW - RT_FLOW_SESSION_DENY [[email protected] source-address="...

props.conf stanza:

[juniper:junos:firewall]
TRANSFORMS-<juniper-fw-host> = <juniper-firewall-host-override>

transforms.conf stanza

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

I tested the REGEX with REX in a search like this:

index=sec_fw sourcetype=juniper:junos:firewall | rex "\s(?P<host2>\w+\.\w+\.\w\d+-*\d*)\s"

And it works, the correct field/value pair appears in the field pane.
My indexing time field extraction should replace the syslog host name with "lvs.fwc.p1087-34".
I read and read the RTFM, followed the instructions to the letter, and yet, it just won't work.
Any help will be very much appreciated.

1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

sansay
Contributor

I must be angle-brackets blind.
I have looked at the stanzas many many time over the last 4 hours, and never noticed them.
Thank you so much.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Reuse: Word-play

This puzzle series (first published here) is based on analysing words.Some of the statements in your puzzle ...

Tuning Edge Processor in Hybrid Mode: What Limits Each Data Path

When you run Edge Processor in hybrid mode, alongside a Splunk heavy forwarder on the same host, two questions ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...