Getting Data In

Syslog-ng filter filter props/transforms for data routing

Splunk_citizen
Explorer

Hello Splunkers,

Earlier we were using central syslog-ng server to capture all /var/log/messages from hosts now we have installed UF on unix servers and we are going with UF monitoring instead of syslog route
We dont want capture all events in /var/log/messages, below ones is previous filters in syslog-ng.conf files in host.
So now my question is how i can write a transform/props to reflects same like below syslog-ng.conf file and deploy locally in forwards i order to capture /var/log/messages to monitor.

# destinations
destination d_sshpam { file("/var/adm/syslog/sshd.log"); };
destination d_sshpamb { file("/var/adm/syslog/sshd.log"); };
#destination d_sapdelete { file("/dev/null"); };
#destination d_sapruntime { file("/dev/null"); };
#destination d_emfbackground { file("/dev/null"); };
destination d_messages { file("/var/adm/syslog/syslog.log"); };

destination d_logserver {syslog("xx.xx.xx.xx" transport(tcp)); };

log { source(s_local); filter(f_sshpam); destination(d_sshpam); flags(final); };

log { source(s_local); filter(f_sshpamb); destination(d_sshpamb); flags(final); };

#log { source(s_local); filter(f_sapdelete); destination(d_sapdelete); flags(final); };
#
#log { source(s_local); filter(f_sapruntime); destination(d_sapruntime); flags(final); };

#log { source(s_local); filter(f_emfbackground); destination(d_emfbackground); flags(final); };

log {
source(s_local);

destination(d_messages);
destination(d_logserver);
};
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi Splunk_citizen,
you can filter data on Indexers or on Heavy Forwarders.
To filter events you can see at http://docs.splunk.com/Documentation/Splunk/7.1.2/Forwarding/Routeandfilterdatad
In easy words, you have to:

  • identify regexes of the logs you want to index or to discard,
  • indert in props.con the following row in the sourcetype of your syslog:

    [my_sourcetype]
    TRANSFORMS-filter = setnull,setparsing

  • insert in transforms.conf the following rows (beware that the stanza's names are the same of TRANSFORMS command:

    [setnull]
    REGEX = .
    DEST_KEY = queue
    FORMAT = nullQueue
    [setparsing]
    REGEX = my_regex
    DEST_KEY = queue
    FORMAT = indexQueue

Bye.
Giuseppe

0 Karma

Splunk_citizen
Explorer

Thank you Casello,

That is the syslog file and not sure what kind of regex i have to build.

0 Karma
Get Updates on the Splunk Community!

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...