topic props.conf w/ Regex ? in Splunk Search

props.conf w/ Regex ?

tmarlette — Tue, 30 Apr 2013 21:50:04 GMT

is it possible to exclude specific results in a field from the search in the props.conf? I suppose more specifically on the backend?

Currently I am using a series of regex statements to exclude some values such as:

< mysearch > | regex < field1 >!= < value > | regex < field1 >!= < value >

is there a better way to do this?

Re: props.conf w/ Regex ?

kristian_kolb — Tue, 30 Apr 2013 23:44:58 GMT

Not sure what you mean, really.

With props/transforms you can filter out events so they never get indexed. You can also set up search time field extractions and field aliases, for example.

However, you can't filter out search results the way your search example describes.

Also, why use | regex field != value ? Unless you have some pattern matching to do, you could stick it before the first pipe as field != value or use | search field != value. But perhaps these are newly eval'ed fields of a complicated nature.

Perhaps if you provide some sample events you'd be able to get better help.

Re: props.conf w/ Regex ?

tmarlette — Wed, 01 May 2013 16:13:45 GMT

In this case i'm looking in web logs. Some of the fields periodically (such as useragent) end up with a null value because of internal machine queries. this throws off some our analytics.

I have to keep the events, simply because they are web events, and the values change. not everything that connects to our web environment has a null value for the useragent field.

I'm looking to see if there is a way that I can simply exclude the 'null' results on the back end, as opposed to doing it in the search query?

Re: props.conf w/ Regex ?

tmarlette — Wed, 01 May 2013 18:33:59 GMT

I've tried to use your suggestion of

field!=value however it is not taking. I assume my syntax is wrong. this ONLY returns the results that I don't want to see.

This is my search string:

sourcetype=www source=< mysource > hck!=health hck!=Health

Re: props.conf w/ Regex ?

kristian_kolb — Wed, 01 May 2013 18:55:36 GMT

I guess that hck is an extracted field. Post a few events, and describe which ones you want to filter out of the search results, and why (i.e. on what criteria)

Re: props.conf w/ Regex ?

kristian_kolb — Wed, 01 May 2013 20:05:26 GMT

Hmm, if you just want to ensure that the user_agent is not null, I guess you could search for;

sourcetype=www user_agent=*

This will only return events that contain the field user_agent, and where it has a non-null value. Of course you can add more fields like referer=* or clientip=*

Re: props.conf w/ Regex ?

tmarlette — Mon, 13 May 2013 13:13:04 GMT

That's OK... I just used the regex statements to filter out all of the unwanted events in the search itself. It doesn't seem like there is a way to do it in props / transforms.conf

Thank you !