Splunk Search

Sequentially search a time range by multi-hour blocks, output each to csv

jvesrc
New Member

Hi All!

Here's my scenario: I'm searching 24 hours worth of data, but due to load I can only search in 4 hour increments (very large dataset). I table the results then dedup the data. After I output to a CSV file for each increment.

I'm using an accelerated datamodel for this normally but having to get weeks worth of data in 4h increments is painful. But is it possible to define a search range (start=7/4/2018:00:00:00, end=7/4/2018:23:59:59), then define the time increment (4h), maybe even max concurrent searches set to 1 or 2.

Now with that, the single search will perform 6 searches total in 4 hour increments, with each outputting to csv.

I've been doing some research on this and looks like I may be able to accomplish this with "gentimes" and "map", but I can't get that working at all so not sure if I'm doing it right. My attempt at using these is below (not with the datamodel, but I have the equivalent datamodel search handy if I can use it in here):

| gentimes start=7/4/2018:00:00:00 end=7/4/2018:23:59:59 increment=4h 
| map maxsearches=1 search="search earliest=$starttime$ latest=$endtime$ index=something host=something action!=allowed <other search criteria> | dedup <my fields> | table <my fields> | outputcsv [ | stats count | addinfo | eval filename=strftime(info_min_time, "filename_%d_%m_%y_%H_%M_%S") | return $filename]"

This gives me "no results found" each time. If I run the basic search then I get my results output to csv just fine... Any help would be greatly appreciated!

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...