Knowledge Management

Summary Indexing - include all interresting fields

apgersplunk1
Explorer

The use case I am after is to build a summary index that includes all interesting fields (system TAs and keys from lookups) just like they appear when issuing the search in the search bar. The console searches include all normalized fields from TAs such as the alias of dest_ip->dest as well as my lookup key/val pairs but these fields are not preserved in the summary index.

I am using lookups to sift through very large datasets and the resultant matches are a very very small percentage of the original dataset. This subset, based on identification by the lookups, is what I want to work with. MY_FIELD1 tells me that there was a match then the lookup adds another 20 key/value pairs of context (MY_FIELD2, MY_FIELD3, etc).

Example scheduled search every 15 min:

 MY_FIELD1=* earliest=-30m latest=-15m|fields dest,src,MY_* | collect index=summary

RESULT from search bar - results include MY_* fields, dest, src, etc.

RESULT in index (summary) - includes only host, source, sourcetype, index, linecount, splunk_server, and timestamp. It is missing MY_* fields, dest, src, etc.

In summary - I am trying to build an index of events that include all normalized fields and lookup context such that I can search very quickly through this small subset of data rather than having to repeatedly search by applying lookups against a gigantic dataset. Thanks in advance.

Tags (1)
0 Karma

apgersplunk1
Explorer

Decided to use accelerated data models.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

What’s New in Splunk AI: Volume 02

Welcome to the second edition of “What’s New in Splunk AI” where we look at the latest and greatest updates, ...

Splunk App Dev Quarterly Roundup: AI, Agents, and Innovation!

Another quarter, another wave of innovation. From complex integrations to pushing the limits ...

Federated Search for Dynamic Data Self Storage Is Now Generally Available on Splunk ...

 Splunk is excited to announce the General Availability of Federated Search for Dynamic Data Self Storage ...