Splunk Search

How to write one events to multiple indexes

kmccarthy
New Member

I have many many events and they are all useful but there is a subset that is most important. I want to keep all events in the primary index and then for efficiency (dashboards and searches) create a separate index with just the important events.
(I use summary indexes for top level charts which works great but when I drill down from the chart retrieving the events is very slow - which is why I want to create the intermediary index with just the important events).

I've been trying variations of props.conf and transforms.conf to write the important events to a second indes.

props.conf

[source::/var/log/ftauditevents]

TRANSFORMS-index = ImportantEvents

transforms.conf

[ImportantEvents]

REGEX =

DEST_KEY = _MetaData:Index

FORMAT = important_index

The problem is that the important events are not written to the primary index - and I still want the primary index to contain all events for other purposes.

Any suggestion?

Tags (1)
0 Karma

FritzWittwer_ol
Contributor

I have the same requirement here, I know we will use more licences but we need some events in more than one index, did you find a solution?

0 Karma

GKC_DavidAnso
Path Finder

If we write the data to two indexes, you will be using more license than necessary.

In your situation, I would consider:

  • Can't I just move the important data to a different index and search across them both when I want all the data (index=important OR index=hohum)?
  • Why is the drilldown search so slow? Can I make the search more efficient? (Have you ordered your search so that it is in the order that excludes the most items first? Have you used |fields to avoid extracting fields you don't need?)

I hope that helps.

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...