Hey all. I need help to selective forward (on a HF) from a log file that is being monitored by a UF. I only need to forward lines that contain the exact words "Read line". I've tried the below confs but the HF is still forwarding all lines that are written to the log.
props.conf
[dcs_event]
TRANSFORMS-routing = dcs_allow,dcs_drop
transforms.conf
[dcs_allow]
DEST_KEY = queue
REGEX = (Read line)
FORMAT = indexQueue
[dcs_drop]
DEST_KEY = queue
REGEX = .
FORMAT = nullQueue
Hi @Adevill,
please try inverting the commands in props.conf
[dcs_event]
TRANSFORMS-routing = dcs_drop, dcs_allow
You have to put before the one containing all the logs (dcs_drop) and then the one to take the selected logs (dcs_allow) as described at https://docs.splunk.com/Documentation/Splunk/8.1.3/Forwarding/Routeandfilterdatad#Keep_specific_even...
the problem is only on props.conf, instead it isn't relevant in transforms.conf.
Ciao.
Giuseppe
Ciao Giuseppe @gcusello
I've made the change you indicated but all the lines are still being forwarded. Any other advice?
Regards
Anton
Hi @Adevill,
let me understand:
is it correct?
One quick question: you have a multi line event or a single line event?
The above approach is correct in single lines events, if instead you have a multi line events and you want to take only a part of the event, it's different.
Could you share a sample of your logs (both lines to take and likes to discard)?
Ciao.
Giuseppe
Ciao @gcusello
Yes you are correct with all 3 points. The log file is being written to continuously, so I'm not sure if that will be seen as a single or multline event. I've used the below link for testing:
Hi @Adevill,
they seems to be single line events, so the configuration seem to be correct!
It shouldn't be useful, but, please try this regex in transforms.conf:
REGEX = \s+Read line:
Another stupid question: are you sure that those logs pass through the HF?
Try to put the same props.conf and transforms.conf also on Indexers.
Ciao.
Giuseppe
Ciao @gcusello
I've tried both your suggestions (change the regex and put the conf files at indexer) but all the lines are still being indexed. Any other suggestions I can try?
Hi @Adevill,
at this point, try the more strange things: are you sure that the sourcetype of your logs is "dcs_event"?
Ciao.
Giuseppe
Hi @gcusello
Yes, I'm sure, I've also double checked it and the spelling of everything.
Hi @gcusello
As requested:
[monitor://C:\Program Files (x86)\ABB Symphony Plus\Operations\History\PlantConnect.SYS\Debug\PlaCoEventImporter\Administrator_PlaCoEventImporter.log]
disabled = 0
source = dcs
sourcetype = dcs_event
Hi @Adevill,
it's really strange because:
[dcs_event]
TRANSFORMS-routing = dcs_drop, dcs_allow
The only hint I can give is check again the above chain: something could be different.
Ciao.
Giuseppe
@gcusello ,thank you for all the assistance. I'll go back and check everything everything.
Hi @Adevill,
I'm sorry to be not able to help you more: it's really strange I did many of these condifugrations without any problem.
Check again every step and surely there's a little particular the will solve the problem.
Ciao.
Giuseppe