Splunk search command time modifiers not working

[matthewg]

I want to strip certain results by time from my search. I eventually plen to place a dedup command between the first and second searches, however I am running into issues with the earliest and latest modifiers on search in the second search. The following 3 searches work fine and return results throughout the week:

host=x
host=x earliest=-7d
host=x earliest=-7d | search *

But these searches return no results: even when there are events in the listed time frame.

host=x | search host=x earliest=-7d
host=x | search host=x earliest=-4d

Does anyone have any idea why? I would like to strip off search results based on time in the second search but it doesn't seem to work.

[bowesmana]

With your last two searches, you are effectively specifying 2 search time windows in the same search, so this search

host=x | search host=x earliest=-7d

is saying

1. Look for host=x with the time range specified by the time range picker
2. Within the results of (1), look for host=x over the last 7 days

The earliest=7d is effectively redundant as the only data that gets passed to the 2nd part of the search is the data from the first part.

Your first "host=x" is an implicit search command, so when it hits the first | pipe, that next part of the pipeline only gets data from the first search. You are effectively doing

search host=x | search host=x earliest=-7d

The construct 

search | search is an unnecessary construct. If you want to use multiple search criteria, then do something like

host=x earliest=-7d latest=-6d OR 
(host=x earliest=-4d latest=-3d) OR 
(host=x earliest=-2d latest=-1d)

[matthewg]

Well the issue is I want to run more commands in between: like this for instance:

host=x field=abc earliest=-7d
| dedup field2 sortby +_time
| search * earliest=-1d

The point of this is to remove all events from today's search where there is an event with fields matching the deduped fields that is older than 1 day. This way I will see events that have happened today, but not in the last 7 days, but may or may not have happened before that.

host=x field2=a time=today	drop
host=x field2=a time=yesterday	drop
host=x field2=b time=today	keep
host=x field2=c time=yesterday	drop

[bowesmana · Accepted solution]

You don't need dedup/sort, just do this

...your search... earliest=-7d
| stats count max(_time) as latest min(_time) as earliest by host field2
| eval today=relative_time(now(), "@d")
| where earliest>=today AND latest>=today

So, your effectively saying you only want events in your result set from data over the last 7 days, where all events for host/field2 have come in today.

Depending on what you need from the result set, you may need to add more things to your stats statement or if you want to retain the _raw message, change the stats to eventstats. 

Note that different commands have different performance implications - stats will be the best option and unless you really need _raw and all possible data from remaining events, it's good to try to use stats. You can use stats list() or values() to collect wanted fields, depending on how many events you may get.