Getting Data In

simple index segregation ?

Vtesse
Engager

Hi.

I'm trying to find a quick and simple way to separate my incoming cisco syslogs into different indexes. For complicated, and dull reasons we can only really use the udp:514 listener.

The best I've come up with so far is defining a number of different transforms in the props.conf, then use regex to define which index they go to. unfortunately my splunk and regex skills are currently rather weak and it's refusing to work.

So far I've got to here:
~~ props.conf ~~

[syslog]
TRANSFORMS-routing = generic-routing, MPLS-routing, Office-routing

~~ transforms.conf ~~

[MPLS-routing]
SOURCE_KEY = MetaData:Host
REGEX = 10\.10\.\d+
DEST_KEY = _MetaData:Index
FORMAT = MPLS

[Office-routing]
SOURCE_KEY = MetaData:Host
REGEX = 10\.100\.\d+
DEST_KEY = _MetaData:Index
FORMAT = office

[generic-routing]
DEST_KEY = _MetaData:Index
FORMAT = syslog

Any tips or thoughts?

0 Karma

Vtesse
Engager

apologies for not getting back to you, but I eventually got this working in a similar method to above.
I ended up going for a [source::udp:514] thingy in props.conf then regex'ing in transforms as we discussed.
the trick was, as usual, getting the regex to work. ended up with the regex below to match the basic date/timestamp format of "MMM DD HH:MM:SS I{P OR Hostname}". pretty simple stuff but invaluable nonetheless.

[office_routing]
REGEX = (\w+\ [0-9]{1,2}\ \d+:\d+:\d+ 10.100.[0-9]{1,3}.[0-9]{1,3})|(\w+.domain.ltd)
DEST_KEY = _MetaData:Index
FORMAT = office

And it works perfectly
thanks.

0 Karma

kristian_kolb
Ultra Champion

One thing that could mess things up, and possibly be the source of your problems is that incoming syslog (i.e. with the sourcetype set to syslog) data goes through a TRANSFORM for setting the host value based on the content of the events. Have a look at the $SPLUNK_HOME/etc/system/default/props.conf and transforms.conf to see how this works.

If this happens before your TRANSFORMS the host would not be there, or rather may not have the values you expect.

You can skip the SOURCE_KEY setting (default is the _raw event) and create your regexes to match a timestamp followed by 10.10.* or 10.100.* etc.

Also, you may need to rename the the transforms so they use underscores instead of hyphens;

generic_routing
MPLS_routing
Office_routing

Also, you may have to specify a REGEX = . for the [generic_routing]. (see new spelling)

Hope this helps,

Kristian

kristian_kolb
Ultra Champion

Try the bottom two things first. (REGEX = . and underscores)

/K

0 Karma

Vtesse
Engager

Would it make more sense to use [source::udp] in the props.conf instead of [syslog] ?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...