Getting Data In

Split syslog input into multiple indexes

hollow
Explorer

I'm trying to split messages that come into splunk via UDP:514 (single input, single sourcetype) into multiple indexes based on a regex that should be applied to the _raw message.

I've tried several suggestions i've found in splunk-base but nothing seems to work 😞

My current configuration looks like this:

inputs.conf

[tcp://8514]
connection_host = ip
sourcetype = syslog

props.conf

[syslog]
TRANSFORMS-index=route-to-index

transforms.conf

[route-to-index]
REGEX = ^<\d+>(app|pubsub|updater)(?:\[(\d+)\])?:\s
FORMAT = index::myindex
WRITE_META = true

The inputs.conf is in etc/system/local/inputs.conf, props and transforms are in a custom app.

The goal is to filter based on the process name for now. The regex definitely matches the messages, but nothing appears in myindex.

1 Solution

dart
Splunk Employee
Splunk Employee

You're pretty close. In your

transforms.conf

[route-to-index]
REGEX = ^<\d+>(app|pubsub|updater)(?:\[(\d+)\])?:\s
FORMAT = myindex
DEST_KEY = _MetaData:Index

Should do the trick.

View solution in original post

dart
Splunk Employee
Splunk Employee

You're pretty close. In your

transforms.conf

[route-to-index]
REGEX = ^<\d+>(app|pubsub|updater)(?:\[(\d+)\])?:\s
FORMAT = myindex
DEST_KEY = _MetaData:Index

Should do the trick.

hollow
Explorer

i could swear that i also had this variant in my configs before. but it seems to work now, thanks a lot! 🙂

0 Karma

zugji
Path Finder

Is this sill working if in inputs.conf an index is defined?
Let's say:

index = main
0 Karma
Get Updates on the Splunk Community!

Dashboard Studio Challenge - Learn New Tricks, Showcase Your Skills, and Win Prizes!

Reimagine what you can do with your dashboards. Dashboard Studio is Splunk’s newest dashboard builder to ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...