Getting Data In

overriding default host field extraction won't work

sansay
Contributor

After spending 4 hours trying to make this work I am out of ideas.
Here is my final setup and it is what I think should work:

Sample event:

Jun  4 09:58:04 10.237.94.1 1 2017-06-04T16:58:03.876Z lvs.fwc.p1087-34 RT_FLOW - RT_FLOW_SESSION_DENY [junos@2636.1.1.1.2.28 source-address="...

props.conf stanza:

[juniper:junos:firewall]
TRANSFORMS-<juniper-fw-host> = <juniper-firewall-host-override>

transforms.conf stanza

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

I tested the REGEX with REX in a search like this:

index=sec_fw sourcetype=juniper:junos:firewall | rex "\s(?P<host2>\w+\.\w+\.\w\d+-*\d*)\s"

And it works, the correct field/value pair appears in the field pane.
My indexing time field extraction should replace the syslog host name with "lvs.fwc.p1087-34".
I read and read the RTFM, followed the instructions to the letter, and yet, it just won't work.
Any help will be very much appreciated.

1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@sansay

Two things to check.

  1. That your props/transforms are configured on the indexer since this is an index-time field extraction
  2. Take out the brackets from your props.conf surrounding juniper-fw-host and juniper-firewall-host-override

Here is an example that should work for you:
inputs.conf (forwarder)

[monitor:///opt/splunkforwarder/logs/test.log]
index=main
sourcetype= juniper:junos:firewall

props.conf (indexer)

[juniper:junos:firewall]
TRANSFORMS-juniper-fw-host = juniper-firewall-host-override

transforms.conf (indexer)

[juniper-firewall-host-override]
DEST_KEY = MetaData:Host
REGEX = \s(\w+\.\w+\.\w\d+-*\d*)\s
FORMAT = host::$1

restart Splunk after configurations are made

sansay
Contributor

I must be angle-brackets blind.
I have looked at the stanzas many many time over the last 4 hours, and never noticed them.
Thank you so much.

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...