Splunk Search

What is a good approach to run a search multiple times in time batches?

Contributor

Hi all,

I am trying to run a search that returns one row of results over a long historical time window on a per hour basis.

Due to investigation method, I need to run this over around almost 60 days.

The search cannot be timecharted or timeboxed via stats because there are subsearches that return fields relevant for that particular hour...

I was thinking of the below search in a scheduled search...

Each time range will have a row with results as well because we have append=true

The problem is, if there are no results than the row is not generated, & as a result, the time lookup is stuck at that time, window & stale..

How do I account for this?

OR

Is there a better approach to run searches multiple times on a time basis. e.g 1 hour.

index=_internal
    [| inputlookup results_and_time.csv 
     | tail 1
    | fields earliestTime latestTime 
    | rename earliestTime as earliest , latestTime as latest 
    | format] 
| addinfo 
| stats values(sourcetype) as sourcetype earliest(info_min_time) as earliestTime , latest(info_max_time) as latestTime 
| eval earliestTime = relative_time(earliestTime,"+1h") , latestTime = relative_time(latestTime,"+1h") 
| outputlookup append=true results_and_time.csv

p.s. Adding the time range manually for the initial run without the inputlookup to add the earliest & latest..

0 Karma

Ultra Champion

Just schedule the search with a time window defined for example: -400h@h till 399h@h and schedule it to run each hour, then the time window automatically shifts each hour. Just pick the offset such that you search over the desired window?

0 Karma

Contributor

In this case we will have to wait for timespan='the number of reporting hours' to cover all the offset windows

or did i miss something in the logic please?

This is spanning 60+ days thus my attempt to auto increment the span for every subsequent search..

0 Karma

Ultra Champion

Yes, that is correct.

0 Karma

Contributor

ah okay.I dont think ops will want to wait that long for my case 😄

This approach is definitely the cleanest way for shorter reporting timespans though!

0 Karma

Path Finder

Hi!
You can try to use gentimes + map:

| gentimes start=-60 increment=1h
| map maxsearches=10000 search="search index=_internal earliest=$starttime$ latest=$endtime$ | stats values(sourcetype) as sourcetypes | eval earliest=\"$starttime$\", latest=\"$endtime$\" "
0 Karma

Contributor

thanks for the answer nryabykh!

This certainly gives me a way to spawn multiple searches off the same base search with incrementing timespans . Which I didnt realise before!

The problem is that The map command spawns them at a go?

or

is it sequential & will only fire off after once the 'map search' is completed for the 'active' row.

0 Karma

Path Finder

As far as I know, the map command handles the input rows sequentially, but there is no any kind of "preview results". You have to wait for the map command to handle all rows before you're able to do anything else within your query. But it doesn't matter if some of map's subsearches returns nothing. In this case there will be just several empty rows in output table.

0 Karma