Splunk Search

Using lookup table to circumvent search time constraints

michaeler
Communicator

I just want a sanity check to see if this is possible before I go through the effort. I am currently restricted to searching back <=90 days in Splunk but have access to the >90 days data in the source database.

To circumvent this restriction I figure I can convert the old data into a lookup table file, set the time range at "all-time", and append to the lookup table.

Has anyone tried this before or is this theoretically possible?

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @michaeler,

don't use a lookup, but a summary index:

you can schedule a daily search that extract the fields you need and stores them in a summary index using the collect command (https://docs.splunk.com/Documentation/SplunkCloud/9.0.2303/SearchReference/collect) giving to this summary index a greater retention, then you can run your searches on the summary index with also better performances.

For more infos see at https://docs.splunk.com/Documentation/Splunk/9.0.5/Knowledge/Usesummaryindexing#:~:text=From%20the%2...

Ciao.

Giuseppe

0 Karma

michaeler
Communicator

@gcusello Thank you for the response.

I think I'll do that but with data right at the -90d limit to retain it before I lose access. But this won't solve my issue of accessing data that is already outside of my -90d restriction (i.e. -1y to -91d).

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @michaeler,

this solution permits to bypass retentio issue, obviously it cannot solve access limitations for your role.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...