Splunk Search

Sequentially search a time range by multi-hour blocks, output each to csv

jvesrc
New Member

Hi All!

Here's my scenario: I'm searching 24 hours worth of data, but due to load I can only search in 4 hour increments (very large dataset). I table the results then dedup the data. After I output to a CSV file for each increment.

I'm using an accelerated datamodel for this normally but having to get weeks worth of data in 4h increments is painful. But is it possible to define a search range (start=7/4/2018:00:00:00, end=7/4/2018:23:59:59), then define the time increment (4h), maybe even max concurrent searches set to 1 or 2.

Now with that, the single search will perform 6 searches total in 4 hour increments, with each outputting to csv.

I've been doing some research on this and looks like I may be able to accomplish this with "gentimes" and "map", but I can't get that working at all so not sure if I'm doing it right. My attempt at using these is below (not with the datamodel, but I have the equivalent datamodel search handy if I can use it in here):

| gentimes start=7/4/2018:00:00:00 end=7/4/2018:23:59:59 increment=4h 
| map maxsearches=1 search="search earliest=$starttime$ latest=$endtime$ index=something host=something action!=allowed <other search criteria> | dedup <my fields> | table <my fields> | outputcsv [ | stats count | addinfo | eval filename=strftime(info_min_time, "filename_%d_%m_%y_%H_%M_%S") | return $filename]"

This gives me "no results found" each time. If I run the basic search then I get my results output to csv just fine... Any help would be greatly appreciated!

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...