I've got a bunch of key-value data, something sorta like this:
I'd like to sort this data into different indexes (for the purpose of different retention times) depending on the value passed to the 'b' key. So, if b=2, send the data to index_retain_for_one_week but if b=6, send the data to index_retain_for_one_month. Ideally, there would be a final condition for values of 'b' that aren't listed. Think of this as an 'else' condition that sends non-matching data for the list of conditions to index_retain_one_day. (All of the index names are just illustrative, like the data.)
Is there a good way to do that? Heck, is it possible? My apologies if this is already answered somewhere - I couldn't find a set of key words that generated an answer. Thanks!
I suppose I can make multiple stanzas of transforms and they are applied in order by the TRANSFORMS statement in props.conf. So yeah, this will work! Thanks!
Okay, this is close but not quite working. I've got something like this in transforms.conf:
DEST_KEY = _MetaData:Index
REGEX = b=[3|30|44|49|21]
FORMAT = special-index
FORMAT = normal-index
And then, in props.conf, I have something like this:
TRANSFORMS-indexsort = special-data, normal-data
There must be something unhappy with the REGEX, because everything ends up in special-index. Hmm.
You forgot to add "REGEX = ." in the normal-data stanza.
[special-data] DEST_KEY = _MetaData:Index REGEX = b=[3|30|44|49|21] FORMAT = special-index [normal-data] REGEX = . DEST_KEY=_MetaData:Index FORMAT = normal-index
Check the REGEX for special-data, may be some spaces that need to be adjusted etc. If you can send some actual sample data, I can try to look at it as well..
I discovered that my brain was backwards. It's not first-match and stop in the listed transforms in props.conf - it runs to the end and the last match is what you get.
Once I got that through my skull, everything works as expected. Thanks much!