Get Splunk to Index data based upon host?

[rickyholland87]

Hello this is a bit of a strange one but I'm hoping someone will be able to help.

I currently have Splunk setup to collect data from a single folder. This folder contains .log files from hundres of PC's. These log files are being collected by Powershell and then moved into this single folder.

I am currently gathering the original Host using regex against the log filename which contains the hostname and the date (for example B0292T01_20150615.log).

The problem is that Splunk is currently putting all the logs into a single index which is causing problems with my intended search. I am using a Transaction to calculate the time duration between a specified start event and a specified end event (using time stamps). The log file does not contain any record of its source, this is only recorded in the filename. This transaction seems to be working but the problem is that as the logs are all put into a single index Splunk is linking events from multiple hosts.

Is there anyway I can either automatically split the .logs into separate indexes based upon host?

Thanks for he help.

Best Regards,

Ricky.

[acharlieh]

To answer your question on if data could be routed to multiple indexes, the answer is yes, and the docs: http://docs.splunk.com/Documentation/Splunk/6.2.3/Indexer/Setupmultipleindexes#Route_specific_events... describes the general way to do that

However you issue is that you want your transactions to only be for single hosts. In this case, assuming you're having the field set properly just add host to the list of fields that must keep the same value. ( and/or restrict your base search to a particular host value)

E.g. Instead of:

foo bar | transaction session

Do:

foo bar | transaction session,host

Or:

foo bar host="baz" | transaction session

[woodcock]

I would not do it that way. When you say you are "gathering host" do you mean that you are "overriding the host field" (which is definitely what you should be doing)? Your source contains both the encoded host and something related to source so just use the rex field=source command to pull out some portion that you can then use like this to discriminate your dataset:

... | rex field=source "something to capture 'sourcething'" | search host=hostprefix* sourcething=sourceprefix* | transaction blah blah