Can forwarder read first few lines, then split logs into different indexes?

[cpraznowski_spl]

Folks…gotta question here:

I have two websites flowing access_combined into the same directory.

Each site needs to go to its own Splunk index.

There is nothing in the filename that will identify the site.

Can the forwarder read the first few lines the file and then send the file the appropriate directory?

Going to post on Answers as well.

Clustered WebSphere server serving up many sites at once, logging centrally.

[martin_mueller · Accepted solution]

If there are identifying features in each event instead of the path then you could use transforms.conf routing to set the index.

[martin_mueller]

You'd need a heavy forwarder if you want to perform this at the source, however your indexers can do this too so you can work with a universal forwarder.

I don't quite understand your second point... are you asking where you need to point to your transforms.conf entry? That'd be in props.conf under a sourcetype, source, or host stanza. In your case I'd lean towards source because you probably don't want to apply the transformation to all apache web logs or all logs coming from the entire host.

[cpraznowski_spl]

Agreed...following this doc:
http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad
This is a great feature, in my case, all logging is dumped to single directory from multiple sites, we'll need to pull the eggs out the basket and send to the respective indexes.
Thanks again!

[martin_mueller]

Don't forget to mark the answer as accepted if your issue has been resolved.

[cpraznowski_spl]

Thank you! Is it true that'd I'd need a heavy forwarder to do this & when transform.conf is utilized and source type is set to access_combined, that Splunk will need to re-instructruted to understand that the source is an apache web log? If the second statement is so, is that simply a setting in props.conf?