Splunk Search

Heavy Forwarder Selective forwarding

Adevill
Loves-to-Learn Lots

Hey all. I need help to selective forward (on a HF) from a log file that is being monitored by a UF. I only need to forward lines that contain the exact words "Read line". I've tried the below confs but the HF is still forwarding all lines that are written to the log.

props.conf

 

[dcs_event]
TRANSFORMS-routing = dcs_allow,dcs_drop

 

transforms.conf

 

[dcs_allow]
DEST_KEY = queue
REGEX = (Read line)
FORMAT = indexQueue

[dcs_drop]
DEST_KEY = queue
REGEX = .
FORMAT = nullQueue

 

 

 

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

please try inverting the commands in props.conf 

[dcs_event]
TRANSFORMS-routing = dcs_drop, dcs_allow

You have to put before the one containing all the logs (dcs_drop) and then the one to take the selected logs (dcs_allow) as described at https://docs.splunk.com/Documentation/Splunk/8.1.3/Forwarding/Routeandfilterdatad#Keep_specific_even...

the problem is only on props.conf, instead it isn't relevant in transforms.conf.

Ciao.

Giuseppe

0 Karma

Adevill
Loves-to-Learn Lots

Ciao Giuseppe @gcusello 

I've made the change you indicated but all the lines are still being forwarded. Any other advice?

 

Regards

Anton

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

let me understand:

  • you inverted the positions in props.conf command,
  • you restarted HF,
  • you continue to receive all the lines from the HF;

is it correct?

One quick question: you have a multi line event or a single line event?

The above approach is correct in single lines events, if instead you have a multi line events and you want to take only a part of the event, it's different.

Could you share a sample of your logs (both lines to take and likes to discard)?

Ciao.

Giuseppe

0 Karma

Adevill
Loves-to-Learn Lots

Ciao @gcusello 

Yes you are correct with all 3 points. The log file is being written to continuously, so I'm not sure if that will be seen as a single or multline event. I've used the below link for testing:

 

https://regex101.com/r/3RmTnm/1 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

they seems to be single line events, so the configuration seem to be correct!

It shouldn't be useful, but, please try this regex in transforms.conf:

 

REGEX = \s+Read line:

 

Another stupid question: are you sure that those logs pass through the HF?

Try to put the same props.conf and transforms.conf also on Indexers.

Ciao.

Giuseppe

 

0 Karma

Adevill
Loves-to-Learn Lots

Ciao @gcusello 

I've tried both your suggestions (change the regex and put the conf files at indexer) but all the lines are still being indexed. Any other suggestions I can try?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

at this point, try the more strange things: are you sure that the sourcetype of your logs is "dcs_event"?

Ciao.

Giuseppe

0 Karma

Adevill
Loves-to-Learn Lots

Hi @gcusello 

Yes, I'm sure, I've also double checked it and the spelling of everything.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

could you share the inputs.conf on the UF that takes those logs?

Ciao.

Giuseppe

0 Karma

Adevill
Loves-to-Learn Lots

Hi @gcusello 

As requested:

[monitor://C:\Program Files (x86)\ABB Symphony Plus\Operations\History\PlantConnect.SYS\Debug\PlaCoEventImporter\Administrator_PlaCoEventImporter.log]
disabled = 0
source = dcs
sourcetype = dcs_event
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

it's really strange because:

  • the sourcetype on UF's inputs.conf is correct (dcs_event);
  • the props.conf on HF is correct
[dcs_event]
TRANSFORMS-routing = dcs_drop, dcs_allow
  • you restarted HF after props.conf modification;
  • The regex in transforms.conf is correct (otherwise you discard all the events),
  • events pass through the HF so they should be parsed by HF.

The only hint I can give is check again the above chain: something could be different.

Ciao.

Giuseppe

0 Karma

Adevill
Loves-to-Learn Lots

@gcusello  ,thank you for all the assistance. I'll go back and check everything everything.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Adevill,

I'm sorry to be not able to help you more: it's really strange I did many of these condifugrations without any problem.

Check again every step and surely there's a little particular the will solve the problem.

Ciao.

Giuseppe

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!