Getting Data In

## Index selection conditional on values in the data

Path Finder

I've got a bunch of key-value data, something sorta like this:

a=1,b=2,c=3,d=4
a=5,b=6,c=7,d=8
a=9,b=2,c=10,d=11
(etc.)

I'd like to sort this data into different indexes (for the purpose of different retention times) depending on the value passed to the 'b' key. So, if b=2, send the data to index_retain_for_one_week but if b=6, send the data to index_retain_for_one_month. Ideally, there would be a final condition for values of 'b' that aren't listed. Think of this as an 'else' condition that sends non-matching data for the list of conditions to index_retain_one_day. (All of the index names are just illustrative, like the data.)

Is there a good way to do that? Heck, is it possible? My apologies if this is already answered somewhere - I couldn't find a set of key words that generated an answer. Thanks!

Tags (3)
1 Solution
Revered Legend
Revered Legend
Path Finder

I suppose I can make multiple stanzas of transforms and they are applied in order by the TRANSFORMS statement in props.conf. So yeah, this will work! Thanks!

Path Finder

Okay, this is close but not quite working. I've got something like this in transforms.conf:

[special-data]
REGEX = b=[3|30|44|49|21]
FORMAT = special-index

[normal-data]
FORMAT = normal-index

And then, in props.conf, I have something like this:

[mysourcetype]
TRANSFORMS-indexsort = special-data, normal-data

There must be something unhappy with the REGEX, because everything ends up in special-index. Hmm.

Revered Legend

You forgot to add "REGEX = ." in the normal-data stanza.

``````[special-data]
REGEX = b=[3|30|44|49|21]
FORMAT = special-index

[normal-data]
REGEX = .
FORMAT = normal-index
``````
Path Finder

Yeah, and I need parens instead of square brackets. Getting there! (Thanks!!)

Path Finder

Hmm, now everything is falling through to normal-data. Time to muck around with it some more.

Revered Legend

Check the REGEX for special-data, may be some spaces that need to be adjusted etc. If you can send some actual sample data, I can try to look at it as well..

Revered Legend

Also, can you do this, in your porps.conf and transforms.conf, change the order of the stanza, so get the normal-data first and special-data after that.

Path Finder

I discovered that my brain was backwards. It's not first-match and stop in the listed transforms in props.conf - it runs to the end and the last match is what you get.

Once I got that through my skull, everything works as expected. Thanks much!

State of Splunk Careers