Getting Data In

Route for different indexes on same file

standias
Explorer

Hi my scenario is: I monitor a file called apps.txt on forwarder side & index it to index=app on Indexer side. I filter out all logs for those that contain App/app & DROP others via nullQueue.

A newer requirement has come up that, I need to now have another index 'alarms' that contain all logs with term alarms from my SAME monitored file i.e apps.txt .(I will send these both logs to same forward server but in different indexes app & alarms). I could just set a regex for alarms send it to different receive group from my props.conf, but how will it then get indexed to 'alarms' index.

Since i have already set a transform for it as retain logs containing term app, index them to app & drop other; how will I further route 'alarms' logs to different index=alarms. Setting a different monitor & different regex will lead to ambiguity??

Here is my current conf:

==============inputs.conf

[monitor://D:\LOGS\apps.txt ]

sourcetype= s_app

index = app

.

.

.

==============props.conf

[s_app]

TRANSFORMS-routing2 = Appredirect

TRANSFORMS-movetonull = notApp

.

.

.

==============transforms.conf

[Appredirect]

REGEX=.*([Aa][Pp][Pp])

DEST_KEY=_TCP_ROUTING

FORMAT=APP

[notApp]

REGEX=^((?![Aa][Pp][Pp]).)*$

DEST_KEY=queue

FORMAT=nullQueue

.

.

.

==============outputs.conf

[tcpout:APP]

server = 172.23.2.2:8100

Tags (1)
0 Karma

dwaddle
SplunkTrust
SplunkTrust

This answer might help you with this:

http://answers.splunk.com/questions/1958/cant-route-forwarded-data-to-different-index/1964#1964

You should be able to apply a transform at the indexer that, based upon a regex of the raw event, changes the index value.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automated Threat Analysis: Available in ES Premier

Automated Threat Analysis: Centralize and Accelerate Phishing Investigations in Splunk Enterprise ...

What’s New in Splunk AI: Volume 02

Welcome to the second edition of “What’s New in Splunk AI” where we look at the latest and greatest updates, ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...