topic Grouping similar fields using search syntax? in Splunk Search

Grouping similar fields using search syntax?

the_wolverine — Sat, 19 Feb 2011 04:53:23 GMT

I have a ton of useragent type fields, like MacOutlook/some_version_x_os_version_etc and Entourage/other_version_x_os_version_etc. How can I group these for reporting so they all show up as MacOutlook and Entourage?

Re: Grouping similar fields using search syntax?

David — Sat, 19 Feb 2011 05:00:48 GMT

You can always use a rex to pull out a subset of the field. For example, if you just want to pull the contents of the useragent field before the /, you could do the following:

YourSearch | rex field=useragent "^(?<GeneralAgent>.*?)/" | stats count by GeneralAgent

If you don't have the useragent already pulled out, you could expand that:

YourSearch | rex field=_raw "UserAgent=(?<GeneralAgent>.*?)/" | stats count by GeneralAgent

You can then move that extraction into your props.conf to make it permanent.

Re: Grouping similar fields using search syntax?

sideview — Sat, 19 Feb 2011 05:54:48 GMT

1) It's a pretty manual technique, but you can do it with eval, if and searchmatch.

For example, this string of evals will create a field called userAgentType whose values are just Entourage, MacOutlook and Other.

<your search>  
| eval userAgentType=if(searchmatch("useragent=*Entourage*"), "Entourage",userAgentType)
| eval userAgentType=if(searchmatch("useragent=*MacOutlook*"), "MacOutlook",userAgentType)
| eval userAgentType=if(isnull(userAgentType),"Other", userAgentType)

and to tack a chart on the end:

<your search>  
| eval userAgentType=if(searchmatch("useragent=*Entourage*"), "Entourage",userAgentType)
| eval userAgentType=if(searchmatch("useragent=*MacOutlook*"), "MacOutlook",userAgentType)
| eval userAgentType=if(isnull(userAgentType),"Other", userAgentType) 
| timechart count by userAgentType

2) Another way you could do it, is to create a bunch of eventtypes that match based on userAgent, and then just do searches with by eventtype on the end. In that scenario though you'd have to be careful not to have overlapping eventtypes, as well as to somehow filter out all of the eventtypes that have nothing to do with 'useragent-ness'