My override index confs are breaking and I cannot find the cause...
Currently I have logs from two sources (A and B) coming in on (port TCP 666) going to one index_A.
Event logs containing: pipe two separate words pipe, like this ---> | Foo Bar | need to go into index_B.
Inputs.conf
[TCP://666]
disabled = 0
connection_host = dns
index = index_A
sourcetype = st_A
To override I created:
in Props.conf
[source::TCP://666]
TRANSFORMS-Indx_B = SEND_TO_Index_B
in Transforms.conf
[SEND_TO_Index_B]
REGEX = |Foo Bar|
DEST_KEY = _MetaData:Index
FORMAT = Index_B
When I edit both confs and restart, I don't receive any conf errors on restart, but any events containing |foo bar| are lost or dropped from both indexes.
I grep for either index in splunkd.log in /opt/splunk/var/log/splunk, but I am not finding any clues.
Am I missing an error in my confs?
Is there a specific log that might identify were the events are going?
Please advise
Thank you
Hi Log_wrangler,
there are some issue to check:
where you have props.conf and transforms.conf?
they must be on Heavy Forwarders (if present) or on Indexers.
It's better to use sourcetype in props.conf
[st_A]
TRANSFORMS-Indx_B = SEND_TO_Index_B
Check the regex in transforms.conf: pipe is a special char for regex:
[SEND_TO_Index_B]
REGEX = \|Foo Bar\|
DEST_KEY = _MetaData:Index
FORMAT = Index_B
Bye.
Giuseppe
Hi Log_wrangler,
there are some issue to check:
where you have props.conf and transforms.conf?
they must be on Heavy Forwarders (if present) or on Indexers.
It's better to use sourcetype in props.conf
[st_A]
TRANSFORMS-Indx_B = SEND_TO_Index_B
Check the regex in transforms.conf: pipe is a special char for regex:
[SEND_TO_Index_B]
REGEX = \|Foo Bar\|
DEST_KEY = _MetaData:Index
FORMAT = Index_B
Bye.
Giuseppe
Thank you, I will try your suggestions and let you know.
still not working, so I removed the pipes, now its just
REGEX = Foo Bar
does that require quotes or anything special because its two words?
Thank you
No, you con use space or \s and you don't need quotes.
Only one additional question:
I remember from your previous question that you overrided also sourcetype, so what's the event's sourcetype now, the old or the new one?
so in props.conf put the one you have or try both.
[st_A]
TRANSFORMS-Indx_B = SEND_TO_Index_B
or
[st_B]
TRANSFORMS-Indx_B = SEND_TO_Index_B
Bye.
Giuseppe
Apparently there was an issue with the logs not flowing from the source device, which I interpreted as I made a fatal config. However the escape | foo bar | works fine.
Still testing a double override, Index and sourcetype. Override Index works fine, wondering if there will be a performance hit if I do two overrides. But I will make that a separate question.
Thank you
fyi, it did retain the old/wrong sourcetype but I will fix that later.
In props.conf, can you try to give sourcetype name and in transforms.conf edit REGEX to \|Foo Bar\|