Splunk Search

Log events are being dropped when applying a condition

Rao_KGY
Loves-to-Learn

Hello Everyone, I have built a Splunk query (shared below) recently & I noticed that when apply search condition App_Name IN (*) its actually drop number of event scanned. 

Like if I execute following query it has 2515 events (not to confused with statistics)... 

index=msad_hcv NOT ("forwarded") | spath output=role_name path=auth.metadata.role_name
| mvexpand role_name
| rex field=role_name "(\w+-(?P<App_Name>[^\"]+))"
| search Environment=* type=* request.path=* App_Name IN (*)
| stats count




But in case I comment App_Name IN (*) condition in line no. 4 then it produced 4547 events.  

index=msad_hcv NOT ("forwarded") | spath output=role_name path=auth.metadata.role_name
| mvexpand role_name
| rex field=role_name "(\w+-(?P<App_Name>[^\"]+))"
| search Environment=* type=* request.path=* ```App_Name IN (*)```
| stats count




My question is how can I save the log events from getting dropped when App_Name IN (*) is in force ? Please note that events which are being dropped, are the ones who don't have "App_Name" in their events. 

Labels (1)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

My question is how can I save the log events from getting dropped when App_Name IN (*) is in force ? 

Like @ITWhisperer said, you didn't explain what you expect to get by saving the dropped events WHEN App_name IN (*) is in force.  Unless you illustrate the desired output - which is an essential part of an answerable question, your question is a simple statement of contradictions.

Now I suspect you do not merely want to contradict yourself.  Let me try mind reading: you want a count of events with App_Name, and a separate count for events without.

index=msad_hcv NOT ("forwarded")
| spath output=role_name path=auth.metadata.role_name
| mvexpand role_name
| rex field=role_name "(\w+-(?P<App_Name>[^\"]+))"
| search Environment=* type=* request.path=*
| eval app_name_or_no = if(isnull(App_Name), "no", "yes")
| stats count by app_name_or_no

 If this is tea leaf is telling, the question has nothing to do with events being dropped.

One more thing, I don't see any point of inserting that search command on the 4th line.  It is much more effective if you throw all filters in index search.  What's wrong with this?

index=msad_hcv NOT ("forwarded") Environment=* type=* request.path=*
| spath output=role_name path=auth.metadata.role_name
| mvexpand role_name
| rex field=role_name "(\w+-(?P<App_Name>[^\"]+))"
| eval app_name_or_no = if(isnull(App_Name), "no", "yes")
| stats count by app_name_or_no

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

If you don't want them dropped, don't include App_Name IN (*), simples! :grinning_face:

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...