Splunk Search
Highlighted

Routing the data to a different Index via Regex

Communicator

Hello,

We have a source ABC sending us logs and are being stored inside an index called alllogs. From that source, we want to separate the events, which contains the field SiteUrl: https://www.abc.com/site/PathologyPHI and send these logs to a new index called newlogs, while the rest of the logs should still go to the Index all_logs.

This is what I've tried so far, but all the logs still land up in the Index alllogs, instead of being routed.. FYI, props and transforms are kept under $SPLUNKHOME/etc/system/local, while the inputs.conf is under the TA, which brings the logs's local directory. Had to do this because in props, there is a setting defined for another sourcetype as well. Any suggestions regarding moving them back to TA as well will be appreciated.

In props.conf, I made the following change:

[abc:management:activity]
TRANSFORMS-routing = abc_logs

In Transforms.conf, I did this:

[abclogs]
REGEX = SiteUrl:.+PathologyPHI
DEST
KEY = MetaData:Index
FORMAT = new
logs

Here is the inputs.conf for the source. There are other inputs too, but this inputs comes with the events, which we want to route. We do not want to route all the events coming off this input, only the ones matching the regex:

[splunktaabcmanagementactivity://Auditabc]
contenttype = Audit.abc
index = all
logs
sourcetype = abc:management:activity
interval = 120
tenantname = ABC
disabled = 0
start
by_shell = false

Any help will be highly appreciated.

Thank you

0 Karma
Highlighted

Re: Routing the data to a different Index via Regex

Legend

Hi @shiv1593,
at first, please share again the regex in transforms.conf using the Code Sample button because it isn't possible to read it and check the regex.

Anyway, where do you have props.conf and trasnforms.conf?
they must be on Indexers or (when present) on Heavy Forwarders.

Ciao.
Giuseppe

0 Karma
Highlighted

Re: Routing the data to a different Index via Regex

Communicator

Hi Giuseppe,

Please find the regex below:

REGEX = SiteUrl:.+PathologyPHI

Props and transforms are in the heavy forwarder under /etc/system/local. That's where I'm trying to route the data from.

Thank you

0 Karma
Highlighted

Re: Routing the data to a different Index via Regex

Legend

Hi @shiv1593,
the regex and your approach seem to be correct.
So please some additional info:

A surely stupid question: did you restarted Heavy Forwarder after update?

Logs are in an Universal Forwarder (read by the inputs.conf in the TA) and they are sent to the HF (where there are props.conf and transforms.conf) and then sent to Indexers, is it correct or there's anything else?

I don't understand what you say: "...setting defined for another sourcetype...".

Ciao.
Giuseppe

0 Karma