Splunk Search

How to write one events to multiple indexes

kmccarthy
New Member

I have many many events and they are all useful but there is a subset that is most important. I want to keep all events in the primary index and then for efficiency (dashboards and searches) create a separate index with just the important events.
(I use summary indexes for top level charts which works great but when I drill down from the chart retrieving the events is very slow - which is why I want to create the intermediary index with just the important events).

I've been trying variations of props.conf and transforms.conf to write the important events to a second indes.

props.conf

[source::/var/log/ftauditevents]

TRANSFORMS-index = ImportantEvents

transforms.conf

[ImportantEvents]

REGEX =

DEST_KEY = _MetaData:Index

FORMAT = important_index

The problem is that the important events are not written to the primary index - and I still want the primary index to contain all events for other purposes.

Any suggestion?

Tags (1)
0 Karma

FritzWittwer_ol
Contributor

I have the same requirement here, I know we will use more licences but we need some events in more than one index, did you find a solution?

0 Karma

GKC_DavidAnso
Path Finder

If we write the data to two indexes, you will be using more license than necessary.

In your situation, I would consider:

  • Can't I just move the important data to a different index and search across them both when I want all the data (index=important OR index=hohum)?
  • Why is the drilldown search so slow? Can I make the search more efficient? (Have you ordered your search so that it is in the order that excludes the most items first? Have you used |fields to avoid extracting fields you don't need?)

I hope that helps.

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

 Prepare to elevate your security operations with the powerful upgrade to Splunk Enterprise Security 8.x! This ...

Get Early Access to AI Playbook Authoring: Apply for the Alpha Private Preview ...

Passionate about security automation? Apply now to our AI Playbook Authoring Alpha private preview ...

Reduce and Transform Your Firewall Data with Splunk Data Management

Managing high-volume firewall data has always been a challenge. Noisy events and verbose traffic logs often ...