topic Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days? in Splunk Search

Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

natefly5 — Mon, 23 Feb 2015 18:55:26 GMT

I am trying to display errors from the last 24 hours that have NOT happened in the last 7 days. I only want to see the "new" errors.

When I run this search, it will compare the two time periods and only removes the errors that happened at both times. So it will display the new errors and the old ones. Any help would be greatly appreciated!

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program]

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

somesoni2 — Mon, 23 Feb 2015 20:12:00 GMT

Why not this

sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" 
NOT [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program] 
|stats count by error, msg, program | table error, msg, program

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

aholzer — Mon, 23 Feb 2015 20:14:59 GMT

Add an eval to both sets that identifies one set as the new and one as the old, and just add a where eval="new". Like so:

|set diff [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-7d@d latest=-1d@d sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program |table error, msg, program | eval label="old"] [search sourcetype=Apps (Hosted="A" OR Hosted="B") earliest=-1d@d latest=@m sub_source="'C'" sub_origin="'D'" |stats count by error, msg, program | table error, msg, program | eval label="new"] | search label="new"

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

dwaddle — Tue, 24 Feb 2015 04:54:56 GMT

This is just a thought, but's let's try this without a subsearch or the set command at all..

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats max(older_than_24) as older_than_24, count by error, msg, program 
| search older_than_24=0
| table error, msg, program

Or we can try this, which is similar but gives you count of errors in the last 24 hours.

earliest=-7d@d latest=@m sourcetype=Apps (Hosted="A" OR Hosted="B")  sub_source="'C'" sub_origin="'D'" 
| addinfo
| eval older_than_24 = if( _time < (info_max_time - 86400),1,0)
| stats count(eval(_time < (info_max_time - 86400))) as  count_older_than_24, count(eval(_time >= (info_max_time - 86400))) as count_last_24 by error, msg, program 
| search count_older_than_24=0
| table error, msg, program

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

natefly5 — Tue, 24 Feb 2015 19:03:36 GMT

Thank you! This works perfectly. My follow up question is very basic but how can I add a count to show how many times each type of error has happened in the last 24 hours? Thanks again!

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

dwaddle — Wed, 25 Feb 2015 04:09:04 GMT

does the second search above do what you're seeking here?

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

natefly5 — Tue, 03 Mar 2015 14:28:02 GMT

Yes it works, thanks a lot for your help!

Re: Comparing two time periods with the diff command, how to display only new errors from the last 24 hours that have NOT happened in the last 7 days?

mfscully — Thu, 22 Sep 2016 10:00:14 GMT

Another way to do this is run a scheduled report at midnight that outputs the error, msg, and program for the last 7 days to a lookup. Then you can run your 24 hour report on demand using the lookup to filter. This is probably best performance.