Getting Data In

Can forwarder read first few lines, then split logs into different indexes?

cpraznowski_spl
Splunk Employee
Splunk Employee

Folks…gotta question here:

I have two websites flowing access_combined into the same directory.

Each site needs to go to its own Splunk index.

There is nothing in the filename that will identify the site.

Can the forwarder read the first few lines the file and then send the file the appropriate directory?

Going to post on Answers as well.

Clustered WebSphere server serving up many sites at once, logging centrally.

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

If there are identifying features in each event instead of the path then you could use transforms.conf routing to set the index.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If there are identifying features in each event instead of the path then you could use transforms.conf routing to set the index.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You'd need a heavy forwarder if you want to perform this at the source, however your indexers can do this too so you can work with a universal forwarder.

I don't quite understand your second point... are you asking where you need to point to your transforms.conf entry? That'd be in props.conf under a sourcetype, source, or host stanza. In your case I'd lean towards source because you probably don't want to apply the transformation to all apache web logs or all logs coming from the entire host.

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

Agreed...following this doc:
http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad
This is a great feature, in my case, all logging is dumped to single directory from multiple sites, we'll need to pull the eggs out the basket and send to the respective indexes.
Thanks again!

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Don't forget to mark the answer as accepted if your issue has been resolved.

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

Thank you! Is it true that'd I'd need a heavy forwarder to do this & when transform.conf is utilized and source type is set to access_combined, that Splunk will need to re-instructruted to understand that the source is an apache web log? If the second statement is so, is that simply a setting in props.conf?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...