Getting Data In

Can I override two keys in one transforms stanza?

lyndac
Contributor

My current situation is that a bunch of files are all being dumped into one directory for the forwarder to monitor and send to the indexers. Based on a field in the data, I route the events to different indexes. These are the current props.conf and transforms.conf which are working.

props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
...
TRANSFORMS-override-ldc=override-ldc
TRANSFORMS-override-jrc=override-jrc

transforms.conf:

    [override-ldc]
    SOURCE_KEY=_raw
    DEST_KEY=_MetaData:Index
    REGEX=fieldname\"\s*:\s*\"LDC.*
    FORMAT=foo_ldc

    [override-jrc]
    SOURCE_KEY=_raw
    DEST_KEY=_MetaData:Index
    REGEX=fieldname\"\s*:\s*\"JRC.*
    FORMAT=foo_jrc

I also need to override the value for the source field based on the exact same REGEX. Can I use the same transforms stanza to update 2 metadata fields, or do I need to have a second transform which uses the same REGEX but overrides source rather than index?

0 Karma
1 Solution

woodcock
Esteemed Legend

Your transforms.conf is fine but use this props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
TRANSFORMS-override-index = override-ldc, override-jrc

View solution in original post

0 Karma

somesoni2
Revered Legend

You would've to add different transforms stanza to override Index and source as the DEST_KEY accepts only single fields.

0 Karma

woodcock
Esteemed Legend

Your transforms.conf is fine but use this props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
TRANSFORMS-override-index = override-ldc, override-jrc
0 Karma

lyndac
Contributor

So by listing the tranforms in one line, does that impact the way splunk executes the transforms? Is there a performance impact? I guess I'm asking why one line instead of two?

0 Karma

woodcock
Esteemed Legend

Only very slight improvement but it is better because it is most clear/correct. You can also more easily control which one comes first by the order in the list.

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...