topic Re Route data coming from Syslog to new index in Getting Data In

Re Route data coming from Syslog to new index

Naa_Win — Tue, 26 Nov 2024 16:57:46 GMT

Hello,

I have below inputs stanza to monitor the syslog feed coming to index=base,

Now we need to filter the out with a specific host names and re route them to new index

monitor:///usr/local/apps/logs/*/base_log/*/*/*/*.log]
disabled = 0
sourcetype = base:syslog
index = base
host_segment = 9

example I have hosts (serverxyz.myserver.com, myhostabc.myserver.com, myhostuvw.myserver.com), now i want to match *xyz* and *abc* and re route to new index. since the old config has /*/ which feeds everything to old index i wanted to add balklist to the old stanza to avoid ingesting to both index.

OLD Stanza :

monitor:///usr/local/apps/logs/*/base_log/*/*/*/*.log]
disabled = 0
sourcetype = base:syslog
index = base
host_segment = 9
blacklist = (*xyz*|.*\/*abc*\/)

NEW Stanza :

monitor:///usr/local/apps/logs/*/base_log/*/*/*xyz*/*.log]
disabled = 0
sourcetype = base:syslog
index = mynewindex
host_segment = 9

monitor:///usr/local/apps/logs/*/base_log/*/*/*abc*/*.log]
disabled = 0
sourcetype = base:syslog
index = mynewindex
host_segment = 9

Re: Re Route data coming from Syslog to new index

gcusello — Wed, 27 Nov 2024 07:34:45 GMT

Hi @Naa_Win ,

let me understand:

you want to send data from abc servers to new index and all the others to the old one, is it correct?

you could try something like this:

monitor:///usr/local/apps/logs/*/base_log/*/*/*/*.log] disabled = 0 sourcetype = base:syslog index = base host_segment = 9 blacklist1 = /usr/local/apps/logs/*/base_log/*/*/*xyz*/*.log blacklist2 = /usr/local/apps/logs/*/base_log/*/*/*abc*/*.log monitor:///usr/local/apps/logs/*/base_log/*/*/*xyz*/*.log] disabled = 0 sourcetype = base:syslog index = mynewindex host_segment = 9 monitor:///usr/local/apps/logs/*/base_log/*/*/*abc*/*.log] disabled = 0 sourcetype = base:syslog index = mynewindex host_segment = 9

Ciao.

Giuseppe

Re: Re Route data coming from Syslog to new index

Naa_Win — Wed, 18 Dec 2024 19:43:38 GMT

Hello. I tried this but i didn't worked.

Re: Re Route data coming from Syslog to new index

isoutamo — Wed, 18 Dec 2024 20:19:57 GMT

Hi
I afraid that you must do this in HF or IDX with props and transforms? The reason for that is those file/directory names. I think that splunk internally expand those in one equal name and for that reason you have only one monitoring stanza not three?
Maybe you could try to put those in different order, put that wildcard only into latest. Use btool to look how splunk see those and in which order it put those. Another tool to check how those are read is use “splunk list inputstatus” that shows which files it has read and which stanza those belongs.
r. Ismo