Syslog Filteration

[novotxms]

We are receiving around 300gigs of syslog data everyday and we want to filter all the logs and index only what the network team wants us to. what is the configuration changes that can help me to achieve this? 

How do I filter all the unnecessary logs from the syslog server?

I just need to index the events, where one of the field says sgt=4

Thanks & Regards,

Manyutej Sanjeev

[scelikok]

Hi @novotxms,

You can try sample below;

https://docs.splunk.com/Documentation/Splunk/8.1.2/Forwarding/Routeandfilterdatad#Keep_specific_even... 

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = sgt\=4
DEST_KEY = queue
FORMAT = indexQueue

[novotxms]

Any other answers please? 

[Vardhan]

Hi,

You can use the below filter in the Syslog config. And it will filter only required logs.

filter test{ match( "sgt=4" value("MESSAGE"));};

If you need to add an additional filter you can just use OR operation.

filter test{ match( "sgt=4" value("MESSAGE")); or match( "provide your keyword" value("MESSAGE"));};

if this answer helps you then upvote it.