Splunk Search

How to write one events to multiple indexes

kmccarthy
New Member

I have many many events and they are all useful but there is a subset that is most important. I want to keep all events in the primary index and then for efficiency (dashboards and searches) create a separate index with just the important events.
(I use summary indexes for top level charts which works great but when I drill down from the chart retrieving the events is very slow - which is why I want to create the intermediary index with just the important events).

I've been trying variations of props.conf and transforms.conf to write the important events to a second indes.

props.conf

[source::/var/log/ftauditevents]

TRANSFORMS-index = ImportantEvents

transforms.conf

[ImportantEvents]

REGEX =

DEST_KEY = _MetaData:Index

FORMAT = important_index

The problem is that the important events are not written to the primary index - and I still want the primary index to contain all events for other purposes.

Any suggestion?

Tags (1)
0 Karma

FritzWittwer_ol
Contributor

I have the same requirement here, I know we will use more licences but we need some events in more than one index, did you find a solution?

0 Karma

GKC_DavidAnso
Path Finder

If we write the data to two indexes, you will be using more license than necessary.

In your situation, I would consider:

  • Can't I just move the important data to a different index and search across them both when I want all the data (index=important OR index=hohum)?
  • Why is the drilldown search so slow? Can I make the search more efficient? (Have you ordered your search so that it is in the order that excludes the most items first? Have you used |fields to avoid extracting fields you don't need?)

I hope that helps.

0 Karma
Get Updates on the Splunk Community!

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI! Discover how Splunk’s agentic AI ...