topic Re: split syslog data from multiple ip addresses into separate indexes in Getting Data In

split syslog data from multiple ip addresses into separate indexes

plj3736 — Mon, 21 Apr 2014 20:28:47 GMT

I have multiple linux hosts sending syslog data (port 514) and want to split the data into different indexes based on ip address. I know I can set this up with each sending to a different port, but expect to have more hosts in future so sending to different ports based on ip address could become confusing.

I created a props.conf with

[192.168.17.3]
sourcetype=abc

[192.168.17.4]
sourcetype=mail

but how do I tell splunk to send data from 192.168.17.3 to index abc?

Re: split syslog data from multiple ip addresses into separate indexes

kheli — Mon, 21 Apr 2014 21:42:15 GMT

Here are the steps to achieve it,

Create props.conf to override sourcettype and index. Sourcetype can not be specified under host stanza as you put above in props.conf.

Assume you are using automatic sourcetyping of the the syslog

props.conf

[host::192.168.17.3]
TRANSFORMS-0force_index_sourcetype = 0force_index, 0force_sourcetype

[host::192.168.17.4]
TRANSFORMS=1force_index_sourcetype = 1force_index, 1force_sourcetype

transforms.conf

assign abc index

[0force_index]
SOURCE_KEY=MetaData:Host
REGEX=^192.168.17.3$
DEST_KEY=_MetaData:Index
FORMAT=abc

assign abc sourcetype

[1force_sourcetype]
SOURCE_KEY=MetaData:Host
REGEX=^192.168.17.3$
DEST_KEY=MetaData:Sourcetype
FORMAT=abc

I have not tested this.

Re: split syslog data from multiple ip addresses into separate indexes

grijhwani — Tue, 22 Apr 2014 00:09:02 GMT

You query appears to be addressed in the answer to be found at http://answers.splunk.com/answers/60972/split-syslog-input-into-multiple-indexes with another variant to be found at http://answers.splunk.com/answers/75939/split-syslog-udp514-from-multi-hosts-to-multi-indexes

However, if you are running on Linux or similar (you don't specify), I would strongly recommend installing running syslog-ng (open-source edition should be good enough) as your syslog server, and configuring THAT to be your point of separation and configure your sources accordingly. The native Splunk syslog service is very limited.

Re: split syslog data from multiple ip addresses into separate indexes

plj3736 — Thu, 24 Apr 2014 17:41:25 GMT

I've tried the suggestion from the first answer. transforms.conf seems to have an issues with the assign statement. Indication is this is not a valid statement. I'm somewhat new to working with props and transforms and really novice with REGEX, therefore found the 2nd answer confusing. I had looked at it before I posted the original question.

I someone can help with why the assign statement doesn't work as noted above, would greatly appreciate.

TIA

Re: split syslog data from multiple ip addresses into separate indexes

grijhwani — Fri, 25 Apr 2014 00:55:24 GMT

If you are commenting on answers you should place your comments against the appropriate answer to facilitate an easier discussion. The "assign" remarks in the first answer are purely conversational, not part of the config text.

Re: split syslog data from multiple ip addresses into separate indexes

grijhwani — Fri, 25 Apr 2014 00:57:31 GMT

Edited the above to make the configuration detail stand out from the conversational text.