Getting Data In

How can I override sourcetype and redirect to another index?

danielwysockiar
Explorer

Hi Guys,
I want to override sourcetype for all events before being indexed and redirect some of those events (those with ERROR) to another index with the overridden sourcetype.

So, I need events to be spread between two indexes: test1 and test2 (with ERROR events) and I need all of the events to have the same access_combined sourcetype.

I use oneshot command to ingest data from a file:

  >splunk add oneshot C://opt/log.txt -index test1 -sourcetype test_sourcetype

and now my props.conf looks like this:

[host::myhost]
LINE_BREAKER = \d+(&)  
SHOULD_LINEMERGE = false
TRANSFORMS = custom_sourcetype
TRANSFORMS = route_notfound

LINE_BREAKER is here because its a oneline log, so I need to break it into events and it works fine.

and my transforms.conf:

[custom_sourcetype]
SOURCE_KEY = _raw
REGEX = .*
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::access_combined

[route_notfound]
REGEX = ERROR
DEST_KEY = _MetaData:Index
FORMAT = another_index

and if I use those transforms seperately they work fine (i switch them off by using # in props.conf) but they do not work together....
How can I do those two things in one step? before data being indexed?

1 Solution

sudosplunk
Motivator

In your props.conf, TRANSFORMS must have a unique name.

Please try this,

props.conf:

[host::myhost]
 LINE_BREAKER = \d+(&)  
 SHOULD_LINEMERGE = false
 TRANSFORMS-override = custom_sourcetype
 TRANSFORMS-route = route_notfound

OR

[host::myhost]
 LINE_BREAKER = \d+(&)  
 SHOULD_LINEMERGE = false
 TRANSFORMS-mywork = custom_sourcetype, route_notfound

View solution in original post

sudosplunk
Motivator

In your props.conf, TRANSFORMS must have a unique name.

Please try this,

props.conf:

[host::myhost]
 LINE_BREAKER = \d+(&)  
 SHOULD_LINEMERGE = false
 TRANSFORMS-override = custom_sourcetype
 TRANSFORMS-route = route_notfound

OR

[host::myhost]
 LINE_BREAKER = \d+(&)  
 SHOULD_LINEMERGE = false
 TRANSFORMS-mywork = custom_sourcetype, route_notfound

danielwysockiar
Explorer

Works like a charm! Thank you very much!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...