topic Re: Monitor all remaining files not specifically matched in Getting Data In

Monitor all remaining files not specifically matched

davidstuffle — Sun, 15 Sep 2019 15:57:47 GMT

We have several syslog-ng collectors with UFs on them. The UF monitors the paths and files that syslog-ng generates that we point it to, but I know there are probably several systems sending syslog data that we are missing. Is there a way to point a UF monitor stanza at the top level file path and tell it to monitor everything not matched elsewhere and send it to a specific index so that we can search that index to see what data we're missing?

Re: Monitor all remaining files not specifically matched

richgalloway — Sun, 15 Sep 2019 17:48:17 GMT

While you probably could monitor * and blacklist the files in other monitor stanzas, I wouldn't advise it. Wide wildcards cause the UF to monitor a ton of files and can be very non-performant.

IIRC, syslog-ng has a default directory for data that does not match any rule. I suggest you have your UFs monitor that directory then create an alert to let you know something is in there requiring attention.

Re: Monitor all remaining files not specifically matched

davidstuffle — Mon, 16 Sep 2019 01:08:16 GMT

yeah, it probably would make more sense to do it in the syslog-ng config. I could run all the input through filters with "final" flags sending the matched data to their respective folders and then whatever is left goes to a "not_matched" folder or something.

Re: Monitor all remaining files not specifically matched

frobert — Mon, 16 Sep 2019 06:43:56 GMT

Hi, recent versions of syslog-ng Premium Edition can send log messages to Splunk HEC directly. syslog-ng also has a wildcard file source to monitor files and directories for log messages.