Getting Data In

Can I override two keys in one transforms stanza?

lyndac
Contributor

My current situation is that a bunch of files are all being dumped into one directory for the forwarder to monitor and send to the indexers. Based on a field in the data, I route the events to different indexes. These are the current props.conf and transforms.conf which are working.

props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
...
TRANSFORMS-override-ldc=override-ldc
TRANSFORMS-override-jrc=override-jrc

transforms.conf:

    [override-ldc]
    SOURCE_KEY=_raw
    DEST_KEY=_MetaData:Index
    REGEX=fieldname\"\s*:\s*\"LDC.*
    FORMAT=foo_ldc

    [override-jrc]
    SOURCE_KEY=_raw
    DEST_KEY=_MetaData:Index
    REGEX=fieldname\"\s*:\s*\"JRC.*
    FORMAT=foo_jrc

I also need to override the value for the source field based on the exact same REGEX. Can I use the same transforms stanza to update 2 metadata fields, or do I need to have a second transform which uses the same REGEX but overrides source rather than index?

0 Karma
1 Solution

woodcock
Esteemed Legend

Your transforms.conf is fine but use this props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
TRANSFORMS-override-index = override-ldc, override-jrc

View solution in original post

0 Karma

somesoni2
Revered Legend

You would've to add different transforms stanza to override Index and source as the DEST_KEY accepts only single fields.

0 Karma

woodcock
Esteemed Legend

Your transforms.conf is fine but use this props.conf:

[json_input]
MAX_TIMESTAMP_LOOKAHEAD=30
TRANSFORMS-override-index = override-ldc, override-jrc
0 Karma

lyndac
Contributor

So by listing the tranforms in one line, does that impact the way splunk executes the transforms? Is there a performance impact? I guess I'm asking why one line instead of two?

0 Karma

woodcock
Esteemed Legend

Only very slight improvement but it is better because it is most clear/correct. You can also more easily control which one comes first by the order in the list.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...