topic Re: Count events matching a specific string in Splunk Search

Count events matching a specific string

newtosplunk14 — Mon, 23 Aug 2021 06:20:01 GMT

From the logs, I need to get the count of events from the below msg field value which matches factType=COMMERCIAL and has filters.

secondly, extract the filter type used, like in the example below id and extract the string sorts={"sortOrders":[{"key":"id","order":"DESC"}]}.

Using the Splunk query with basic wildcard does not work efficiently. Could you please assist

cf_space_name=prod msg="*/facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=*"

msg: abc.asia - [2021-08-23T00:27:08.152+0000] "GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null

Thanks in advance.

Re: Count events matching a specific string

kamlesh_vaghela — Thu, 02 Sep 2021 04:51:49 GMT

@newtosplunk14

NOTE: AS I FOUND SOME VALUES ARE DOUBLE ENCODED HENCE USED urldecode TWICE.

Can you please try this?

YOUR_SEARCH | eval _raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%22","\"") ,_raw = replace(_raw , "%5B","["),_raw = replace(_raw , "%5D","]") ,_raw = replace(_raw , "%7B","{"),_raw = replace(_raw , "%7D","}") | extract | search factType="COMMERCIAL" | table factType sorts |rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

My Sample Search :

| makeresults | eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null" | eval _raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%22","\"") ,_raw = replace(_raw , "%5B","["),_raw = replace(_raw , "%5D","]") ,_raw = replace(_raw , "%7B","{"),_raw = replace(_raw , "%7D","}") | extract | search factType="COMMERCIAL" | table factType sorts |rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

UPDATED ANSWER.

| makeresults | eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null" | eval _raw = urldecode(_raw) | eval _raw = urldecode(_raw) | extract | search factType="COMMERCIAL" | table factType sorts |rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Re: Count events matching a specific string

PickleRick — Wed, 01 Sep 2021 12:08:33 GMT

Just extract the appropriate fields from the URI and split the parameters into a multivalued field.

| makeresults 
| eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null\""
| rex "\"(?<req>\S+)\s(?<uri>\S+)\"" 
| eval uri=urldecode(uri) 
| rex field=uri "(?<reqpath>.*)\?(?<query>.*)" 
| makemv delim="&" query 
| search query="factType=COMMERCIAL"

Then you can extract the "filters=something" value from the query field and process it apropriately (probably passing it through another urldecode()).