Splunk Search

Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

natefly5
Explorer

I am trying to display errors from the last 24 hours that have NOT happened in the last 7 days. I only want to see the "new" errors.

When I run this search, it will compare the two time periods and only removes the errors that happened at both times. So it will display the new errors and the old ones. Any help would be greatly appreciated!

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program] 
Tags (2)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

This is just a thought, but's let's try this without a subsearch or the set command at all..

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats max(older_than_24) as older_than_24, count by error, msg, program 
| search older_than_24=0
| table error, msg, program

Or we can try this, which is similar but gives you count of errors in the last 24 hours.

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats count(eval(_time < (info_max_time - 86400))) as  count_older_than_24, count(eval(_time >= (info_max_time - 86400))) as count_last_24 by error, msg, program 
| search count_older_than_24=0
| table error, msg, program

View solution in original post

mfscully
Explorer

Another way to do this is run a scheduled report at midnight that outputs the error, msg, and program for the last 7 days to a lookup. Then you can run your 24 hour report on demand using the lookup to filter. This is probably best performance.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

This is just a thought, but's let's try this without a subsearch or the set command at all..

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats max(older_than_24) as older_than_24, count by error, msg, program 
| search older_than_24=0
| table error, msg, program

Or we can try this, which is similar but gives you count of errors in the last 24 hours.

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats count(eval(_time < (info_max_time - 86400))) as  count_older_than_24, count(eval(_time >= (info_max_time - 86400))) as count_last_24 by error, msg, program 
| search count_older_than_24=0
| table error, msg, program

natefly5
Explorer

Thank you! This works perfectly. My follow up question is very basic but how can I add a count to show how many times each type of error has happened in the last 24 hours? Thanks again!

0 Karma

dwaddle
SplunkTrust
SplunkTrust

does the second search above do what you're seeking here?

0 Karma

natefly5
Explorer

Yes it works, thanks a lot for your help!

aholzer
Motivator

Add an eval to both sets that identifies one set as the new and one as the old, and just add a where eval="new". Like so:

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program | eval label="old"] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program | eval label="new"] | search label="new"
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Why not this

sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" 
NOT [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] 
|stats count by error, msg, program | table error, msg, program
0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...