Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

[natefly5]

I am trying to display errors from the last 24 hours that have NOT happened in the last 7 days. I only want to see the "new" errors.

When I run this search, it will compare the two time periods and only removes the errors that happened at both times. So it will display the new errors and the old ones. Any help would be greatly appreciated!

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program] 

[mfscully]

Another way to do this is run a scheduled report at midnight that outputs the error, msg, and program for the last 7 days to a lookup. Then you can run your 24 hour report on demand using the lookup to filter. This is probably best performance.

[dwaddle · Accepted solution]

This is just a thought, but's let's try this without a subsearch or the set command at all..

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats max(older_than_24) as older_than_24, count by error, msg, program 
| search older_than_24=0
| table error, msg, program

---

Or we can try this, which is similar but gives you count of errors in the last 24 hours.

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats count(eval(_time < (info_max_time - 86400))) as  count_older_than_24, count(eval(_time >= (info_max_time - 86400))) as count_last_24 by error, msg, program 
| search count_older_than_24=0
| table error, msg, program

[natefly5]

Thank you! This works perfectly. My follow up question is very basic but how can I add a count to show how many times each type of error has happened in the last 24 hours? Thanks again!

[dwaddle]

does the second search above do what you're seeking here?

[natefly5]

Yes it works, thanks a lot for your help!

[aholzer]

Add an eval to both sets that identifies one set as the new and one as the old, and just add a where eval="new". Like so:

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program | eval label="old"] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program | eval label="new"] | search label="new"

[somesoni2]

Why not this

sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" 
NOT [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] 
|stats count by error, msg, program | table error, msg, program