Routing to nullQueue based on extracted field

[Paolo_Prigione]

Hi, is it possible to route events to nullQueue based on the value found in a field generated by a csv lookup?

I am facing a SAP audit sequential file which:

1. (on a LWF) is read through a scripted input, which also splits it in lines;
2. has field extraction based on field position and length;
3. field "user_name" is used to lookup a csv file which provides the field "user_role".

I'd need to only index log lines which have a particular value of user_role, but since that value is not present in _raw, I am not able to write the REGEX in the transforms.conf stanza.

Any idea would be greatly appreciated, thanks

[hulahoop]

Hi Paolo, did you ever solve this problem? If not, since you are collecting this data via a scripted input, why not add the lookup capability on the user_name field in the same script? The script could either augment _raw with the user_role, or only write events for roles are you interested in. The 2nd option saves Splunk the trouble of having to apply index-time filtering altogether and maybe some CPU cycles.

[Paolo_Prigione]

Thanks for the suggestion. At the time I didn't have the skills to set that up. I had solved the problem at search time: the volume was not that bad.

[dwaddle]

Sorry Paolo, I don't think it's possible. According to http://www.splunk.com/base/Documentation/latest/Admin/Indextimeversussearchtime lookups are applied at search time, not index time. The nullQueue routing would have to occur at index time in order to be effective.

[Paolo_Prigione]

Thank you Dwaddle; I actually knew it but was hoping on some creative ways to direct inputs to the parsingQueue and then to the nullQueue prior to indexing.