Splunk Search

Question about timemodifier


alt textHi!

I would like to ask about the timemodifier.

I have a following search including subsearch,

[ search index=hoge _index_earliesst=-1d@d _index_latest=@d
| stats
earliest(start) as earliest
latest(stop) as latest by field
| eval earliest=substr(earliest,5,2) . "/" . substr(earliest,7,2) . "/" . substr(earliest,1,4) . ":" . substr(earliest,9,2) . ":" . substr(earliest,11,2) . ":" . substr(earliest,13,2)
| search conditionA
| eval latest=substr(latest,5,2) . "/" . substr(latest,7,2) . "/" . substr(latest,1,4) . ":" . substr(latest,9,2) . ":" . substr(latest,11,2) . ":" . substr(latest,13,2)
| fields field earliest latest
| format "(" "(" "" ")" "OR" ")"

My purpose is to search the events that meets the conditionA that were indexed the previous day
and pass the earliest and latest time of each field to the main search.

However, when the number of events should the main search returns are 5000 , it scans more number of events.

For example,
field earliest latest
fieldA 1/25/2014 00:00 1/25/2014 01:00 3 records exists
fieldB 1/25/2014 02:00 1/25/2014 02:00 5 recoreds exists
fieldC 1/26/2014 00:00 1/26/2014 01:00

  • my latest event in this record is 1/25/2014 01:50:00

if I expect the subsearch to return (fields="fieldA" earliest="1/25/2014:00:00" latest="1/25/2014:01:00") OR ( field="fieldB" earliest="1/25/2014:02:00" latest="1/25/2014 02:00") , I expect the main search to scan only 8 records, But it seems that it scans the event more than I expect.

Is the timemodifier not working corrctly if you concatenate with OR's?

I have added a screen shot where the scanned events are increasing although the mathing events are finished.


Tags (2)
0 Karma


Right - based on your comments, you're probably looking for the map command. Consider these two simple searches:

index=_internal (earliest=-30d@d latest=-29d@d) OR (earliest=@d latest=now) | bin _time span=1d | stats count by _time

That's roughly what you're doing now, tell the search to OR two timeranges... and as you observed, this scans the events in between rather than skipping to the past instantaneously. On my PC, this takes about four seconds.

| stats count | eval times="-30d@d|-29d@d,@d|now" | makemv delim="," times | mvexpand times | makemv delim="|" times | eval starttime=mvindex(times,0) | eval endtime=mvindex(times,1) | map search="search index=_internal earliest=$starttime$ latest=$endtime$ | bin _time span=1d | stats count by _time"

This achieves the same thing, but in a different way. The first bit before map creates two events with fields starttime and endtime, which are used by map to run two searches based on those timeranges. The result is the same as in the earlier search, but my PC only takes about one second.

This should be applicable to your problem, run the current subsearch first and pass into map the earliest/latest times as well as the condition.

0 Karma


Not quite sure about a thousand... map will run a search for each event it's given, so it would run a thousand searches. By default it stops after ten, so running a thousand may or may not break things. Just give it a shot...

While you're studying map, take a look at localize - maybe that could simplify your subsearch for building the timeranges.

0 Karma


Thanks martin!

I have to do some studying with this though. By the way would this work with about thousand set of following conditions( all concatenated with OR)?

(field=hoge1 earliest=xxxx latest=xxxx)

0 Karma



So it will be like,

(field=hoge1 earliest=xxxx latest=xxxx) OR (field=hoge2 earliest=xxxx latest=xxxx) OR ..... and so on.

0 Karma


Good, then I don't need to think about that 🙂

Are you basically trying to do this: "Run a subsearch, get a bunch of results, use each result to build a timerange and possibly further filters, and run a search off that"?

0 Karma


Hello martin.

Yes. The subsearch returns the set of
earliest time in relative time format
latest time in relative time format

format command is working fine.

It seems that the time modifier is not working as expected in each AND boolean expression. Maybe putting to much time modifier confuses the search...

0 Karma


Does the subsearch return what you expected? (Check search inspector)

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!