topic Re: Using transforms to divide monitor input to multiple indexes in Getting Data In

Using transforms to divide monitor input to multiple indexes

Mr_Robaloba — Wed, 23 Mar 2011 18:28:59 GMT

I am trying to filter a log file coming in via a universal forwarder (both installs are 4.2) so that messages containing text X go into index A with sourcetype A, messages with text Y go into index B with sourcetype B, everything else goes to index C with sourcetype C.

The following is inteded to pick out anything with "HostA" and direct it to index hosta_gen, then pick out anything with "HostA Nagios:" in it and send it to the hosta_nagios index, anything else should end up in hosta_cisco.

In props.conf:

[source::/var/log/MyLog.log]
TRANSFORMS-sortIndexes = index_host_messages, sourcetype_host_messages, index_nagios_messages, sourcetype_nagios_messages

In transforms.conf

[index_host_messages]
REGEX = HostA
DEST_KEY = _MetaData:Index
FORMAT = hosta_gen

[sourcetype_host_messages]
REGEX = HostA
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::syslog

[index_nagios_messages]
REGEX = HostA\snagios:
DEST_KEY = _MetaData:Index
FORMAT = hosta_nagios
DEFAULT_VALUE = hosta_cisco

[sourcetype_nagios_messages]
REGEX = HostA\snagios:
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::nagios_log
DEFAULT_VALUE = cisco_syslog

Messages with "HostA" and not "HostA nagios" are ending up in hosta_cisco as type cisco_syslog sourcetype. How can I make this function correctly?

Re: Using transforms to divide monitor input to multiple indexes

dwaddle — Wed, 23 Mar 2011 19:56:31 GMT

This is similar, and may help... please advise

http://answers.splunk.com/questions/1958/cant-route-forwarded-data-to-different-index/1964#1964

Re: Using transforms to divide monitor input to multiple indexes

Mr_Robaloba — Mon, 28 Sep 2020 09:26:31 GMT

Thanks, it is simillar and I can get it to work to that extent. Data with "HostA nagios" does correctly go to the hosta_nagios index.

The difficulty I am having is applying 2 or more conditions to divide my incomming data to 3 indexes. Currently everything other than "HostA nagios" incorretly goes to the hosta_cisco index. I want to be able to fish out data with just "HostA" (and not "nagios") and put that in the hosta_gen index.