Splunk Search

Count events matching a specific string

newtosplunk14
Explorer

From the logs, I need to get the count of events from the below msg field value which matches factType=COMMERCIAL and has filters. 

secondly,   extract the filter type used, like in the example below id and extract the string sorts={"sortOrders":[{"key":"id","order":"DESC"}]}. 

Using the Splunk query with basic wildcard does not work efficiently. Could you please assist

cf_space_name=prod msg="*/facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=*"  

msg: abc.asia - [2021-08-23T00:27:08.152+0000] "GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null

Thanks in advance.

Labels (2)
0 Karma

PickleRick
Ultra Champion

Just extract the appropriate fields from the URI and split the parameters into a multivalued field.

| makeresults 
| eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null\""
| rex "\"(?<req>\S+)\s(?<uri>\S+)\""
| eval uri=urldecode(uri)
| rex field=uri "(?<reqpath>.*)\?(?<query>.*)"
| makemv delim="&" query
| search query="factType=COMMERCIAL"

 Then you can extract the "filters=something" value from the query field and process it apropriately (probably passing it through another urldecode()).

kamlesh_vaghela
SplunkTrust
SplunkTrust

@newtosplunk14 

NOTE:  AS I FOUND SOME VALUES ARE DOUBLE ENCODED HENCE USED urldecode TWICE.

 Can you please try this?

 

YOUR_SEARCH
| eval _raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%22","\"")
,_raw = replace(_raw , "%5B","["),_raw = replace(_raw , "%5D","]")
,_raw = replace(_raw , "%7B","{"),_raw = replace(_raw , "%7D","}") | extract 
| search factType="COMMERCIAL"
| table factType sorts
|rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

 

 

My Sample Search :

 

| makeresults 
| eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null"
| eval _raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%25","%"),_raw = replace(_raw , "%22","\"")
,_raw = replace(_raw , "%5B","["),_raw = replace(_raw , "%5D","]")
,_raw = replace(_raw , "%7B","{"),_raw = replace(_raw , "%7D","}") | extract 
| search factType="COMMERCIAL"
| table factType sorts
|rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

 

 

Screenshot 2021-09-01 at 5.07.24 PM.png

 

UPDATED ANSWER.

YOUR_SEARCH
| eval _raw = urldecode(_raw)
| eval _raw = urldecode(_raw) | extract 
| search factType="COMMERCIAL"
| table factType sorts
|rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

 

| makeresults 
| eval _raw="msg: abc.asia - [2021-08-23T00:27:08.152+0000] \"GET /facts?factType=COMMERCIAL&sourceSystem=ADMIN&sourceOwner=ABC&filters=%257B%2522stringMatchFilters%2522:%255B%257B%2522key%2522:%2522BFEESCE((json_data-%253E%253E'isNotSearchable')::boolean,%2520false)%2522,%2522value%2522:%2522false%2522,%2522operator%2522:%2522EQ%2522%257D%255D,%2522multiStringMatchFilters%2522:%255B%257B%2522key%2522:%2522json_data-%253E%253E'id'%2522,%2522values%2522:%255B%25224970111%2522%255D%257D%255D,%2522containmentFilters%2522:%255B%255D,%2522nestedMultiStringMatchFilter%2522:%255B%255D,%2522nestedStringMatchFilters%2522:%255B%255D%257D&sorts=%257B%2522sortOrders%2522:%255B%257B%2522key%2522:%2522id%2522,%2522order%2522:%2522DESC%2522%257D%255D%257D&pagination=null"
| eval _raw = urldecode(_raw)
| eval _raw = urldecode(_raw) | extract 
| search factType="COMMERCIAL"
| table factType sorts
|rex field=sorts "\"key\":\"(?<key>[^\"]+)\",\"order\":\"(?<order>[^\"]+)\""

 

 

Screenshot 2021-09-02 at 10.20.48 AM.png

Thanks
KV
▄︻̷̿┻̿═━一    

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>