topic Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf? in Getting Data In

Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

zugji — Wed, 04 Feb 2015 08:23:40 GMT

In our splunk environment, we collect and index all syslog messages from our network elements. Some of the syslog messages we would like to forward to another system by syslog protocol (UDP/514). I know it is possible to route syslog messages to another system. As we get about 1 million syslog messages per day we would like to filter the most of them to prevent flooding the target host.

Is that possible with a saved search scheduling every minute or do I have to use the REGEX value in the transforms.conf configuration file?

Our environment.
Splunk 6.2.1
OS: Solaris X86

Regards,
Christain

Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

MuS — Wed, 04 Feb 2015 09:13:43 GMT

Hi zugji,

using the REGEX in transforms.conf is the way to go.
You could do it by using a saved search and some custom search command, which takes the search result and pushes it out to the other syslog receiver....But, as you can image this will need some heavy coding.

So, using the REGEX in transforms.conf is simple, easy, faster and available right now.

Hope this helps ...

cheers, MuS

Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

zugji — Wed, 04 Feb 2015 09:25:34 GMT

Hello MuS
Thanks a lot for your answer. Our saved search to get the needed messages out is a little bit complicate. I think it would be easier to have a saved search running and doing the job. Here is the search:

index=* NOT sourcetype=stash| regex _raw="Closed telnet|OSAPI-5-CLEAN_TASK:\s*osapi_task.c:(?:.*)cleaning\s*up\s*exited\s*task|SYS-6-CFG_CHG.*?/(\S+)/|bsnConfigurationSavedToNvram|SYS-5-RELOAD|SYS-5-RESTART|#\d+ Session closed|%CONFIG|SYSLOG_CONFIG|Connection logout|user.*connected from|CLM: Logout|configure changed|System restarted|AAA-5-AAA_AUTH_ADMIN_USER: aaa.(.*)[\t ]for[\t ]admin[\t ]user[\t ]'(.*)'|SYS-5-CONFIG_I|SYSTEM_RESET|entering configuration mode|UI_COMMIT|SYS-6-CFG|10HWCM|User \S+ authenticated|Authentication succeeded|Session logged out|CLM: Login|Save config|Successful connection|user:.*command:|VTY login from|exiting configuration mode|User \S+ executed the [\S\s]+ command|VTY logout from"

Regards,
Christian

Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

MuS — Wed, 04 Feb 2015 10:30:43 GMT

This basically just one big regex which could be used in the transforms.conf .... needs testing anyway, so why not use the existing Splunk internal features?

Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

zugji — Wed, 04 Feb 2015 11:59:45 GMT

I will give it a try. Thanks for your input!

Re: Is it possible to schedule a saved search to forward Syslog events or do I have to use REGEX in transforms.conf?

zugji — Thu, 05 Mar 2015 12:36:58 GMT

Finally I found a solution to reach my goal.

First define your search xy and enable summary for the search which is in my environment scheduled every minute.
After that you can search the result by using the summary index.
The events in this summary index has the source name equal to the saved search xy.

savedsearch.conf

[xy]
search=...
action.summary_index = 1
action.summary_index._name = thirdparty

props.conf

[source::xy]
TRANSFORMS-routing = forward_xy

transforms.conf

[forward_xy]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = forward-host

outputs.conf

[syslog:forward-host]
server = a.b.c.d:514

Regards,
Christian