Why my configuration is not working ? (nullQueue Windows)

[MCH2018]

Hi everyone,

First of all i have tried every solution present in splunk answers on this subject but no one solved my issue.

I work in an complicate architecture and I want to filter logs from Stormshield Firewall. Stormshield logs are retrieved using a Syslog that writes .txt files and these files are then monitored by Splunk.
The sending logic is as follows : FileSyslog on HF --> another HF --> IDX. This all works well but now I would like the events containing alarmid=1 in these files not to be indexed.

For this I tried to modify the TA-Stormshield directly on the indexer like this :

props.conf :

[stormshield]
TRANSFORMS-sns = snsnull

transforms.conf :

[snsnull]
REGEX = alarmid=1
DEST_KEY = queue
FORMAT = nullQueue

It didn't work, then I tried to share the TA-Stormshield to the 4 HF involved in the log exchange, not working.
Then I tried to modify the REGEX like this :

REGEX = (\balarmid=1\b)
OR
REGEX = ^.*alarmid=1.*$

Not working again.

After i try another strategy, I removed everything I had written in the TA-Stormshield to write directly to .../etc/system/local by creating props.conf and transforms.conf files.
In these files I tried to use the source of the monitored files like this :

props.conf :

[source::E:\LOG\*.txt] # I tried a lot of different syntax for this stanza without result
TRANSFORMS-sns = snsnull

*transforms.conf: *

[snsnull]
REGEX = alarmid=1
DEST_KEY = queue
FORMAT = nullQueue

My logs look like this and I would like the event containing alarmid=1 not to be indexed :

1. 2020-01-22 00:00:00 User.Alert id=firewall time="2020-01-22 00:00:00" ipv=4 action=block class=protocol alarmid=1 logtype="alarm"
2. 2020-01-22 00:00:00 User.Warning id=firewall time="2020-01-22 00:00:00" ipv=4 action=pass class=protocol alarmid=7 logtype="alarm"

I also tried several solutions present on the splunk answer, without much result.
If anyone has a solution I would be extremely grateful.

Thanks for your help.

[mdsnmss]

Full instances of Splunk send what is "cooked" data. Heavy forwarders are a full instance of Splunk so from the first input where you are monitoring your .txt file you are sending cooked data to the other HF and then on to the IDX. Things such as indexed extractions and filtering need to take place on uncooked data as that is part of the "cooking" process. So the filtering needs to take place at your first HF where the .txt file resides. You can also try setting sendCookedData = falseon the first HF in the process as well.

More info on the types of forwarder data is here: https://docs.splunk.com/Documentation/Splunk/8.0.1/Forwarding/Typesofforwarders#Types_of_forwarder_d...

[MCH2018]

Hi mdsnmss,

I got now an error, ERROR TcpInputProc - Message rejected. Received unexpected message of size=842019376 bytes because of the parameter sendCookedData = false placed on the first HF.

The second HF doesn't receive any more events and closes the connection with the first one, so I removed the sendCookedData parameter.

And unfortunately putting the filter configuration on the first HF does not remove the events containing alarmid.