Getting Data In

Syslog-ng filter filter props/transforms for data routing

Splunk_citizen
Explorer

Hello Splunkers,

Earlier we were using central syslog-ng server to capture all /var/log/messages from hosts now we have installed UF on unix servers and we are going with UF monitoring instead of syslog route
We dont want capture all events in /var/log/messages, below ones is previous filters in syslog-ng.conf files in host.
So now my question is how i can write a transform/props to reflects same like below syslog-ng.conf file and deploy locally in forwards i order to capture /var/log/messages to monitor.

# destinations
destination d_sshpam { file("/var/adm/syslog/sshd.log"); };
destination d_sshpamb { file("/var/adm/syslog/sshd.log"); };
#destination d_sapdelete { file("/dev/null"); };
#destination d_sapruntime { file("/dev/null"); };
#destination d_emfbackground { file("/dev/null"); };
destination d_messages { file("/var/adm/syslog/syslog.log"); };

destination d_logserver {syslog("xx.xx.xx.xx" transport(tcp)); };

log { source(s_local); filter(f_sshpam); destination(d_sshpam); flags(final); };

log { source(s_local); filter(f_sshpamb); destination(d_sshpamb); flags(final); };

#log { source(s_local); filter(f_sapdelete); destination(d_sapdelete); flags(final); };
#
#log { source(s_local); filter(f_sapruntime); destination(d_sapruntime); flags(final); };

#log { source(s_local); filter(f_emfbackground); destination(d_emfbackground); flags(final); };

log {
source(s_local);

destination(d_messages);
destination(d_logserver);
};
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi Splunk_citizen,
you can filter data on Indexers or on Heavy Forwarders.
To filter events you can see at http://docs.splunk.com/Documentation/Splunk/7.1.2/Forwarding/Routeandfilterdatad
In easy words, you have to:

  • identify regexes of the logs you want to index or to discard,
  • indert in props.con the following row in the sourcetype of your syslog:

    [my_sourcetype]
    TRANSFORMS-filter = setnull,setparsing

  • insert in transforms.conf the following rows (beware that the stanza's names are the same of TRANSFORMS command:

    [setnull]
    REGEX = .
    DEST_KEY = queue
    FORMAT = nullQueue
    [setparsing]
    REGEX = my_regex
    DEST_KEY = queue
    FORMAT = indexQueue

Bye.
Giuseppe

0 Karma

Splunk_citizen
Explorer

Thank you Casello,

That is the syslog file and not sure what kind of regex i have to build.

0 Karma
Get Updates on the Splunk Community!

AI for AppInspect

We’re excited to announce two new updates to AppInspect designed to save you time and make the app approval ...

App Platform's 2025 Year in Review: A Year of Innovation, Growth, and Community

As we step into 2026, it’s the perfect moment to reflect on what an extraordinary year 2025 was for the Splunk ...

Operationalizing Entity Risk Score with Enterprise Security 8.3+

Overview Enterprise Security 8.3 introduces a powerful new feature called “Entity Risk Scoring” (ERS) for ...