topic Re: spliting multiple feed that use udp:514 in Getting Data In

spliting multiple feed that use udp:514

jarjoh42 — Fri, 21 Dec 2012 18:30:51 GMT

I have multiple systems reporting over UDP:514. I want to separate the iron port email, Cisco ASA's, iseries as400, and PaloAlto firewall feeds at the global level before sending the data to the separate apps. I currently have this stanza in my etc\system\local\inputs file

[udp://514]
connection_host = ip

index = index_syslog

sourcetype = syslog

I know I need to write a stanza in the props.conf to separate out the individual sources by ip address but am not sure how to do it.

Re: spliting multiple feed that use udp:514

alacercogitatus — Fri, 21 Dec 2012 18:47:48 GMT

You'll probably want an index-time transform to do this. You can chain the transforms, and use REGEX to match which IPs you want to have specific sourcetypes.

props.conf

[syslog] TRANSFORMS-forcesourcetype = cisco_asa_st, iron_port, i_as400

transforms.conf

[cisco_asa_st] REGEX = 192\.168\.0\.1|192\.168\.0\.2 SOURCE_KEY = MetaData:Host DEFAULT_VALUE = cisco_asa DEST_KEY = MetaData:Sourcetype

[iron_port] REGEX = 192\.168\.0\.1|192\.168\.0\.2 SOURCE_KEY = MetaData:Host DEFAULT_VALUE = iron_port DEST_KEY = MetaData:Sourcetype

[i_as400] REGEX = 192\.168\.0\.1|192\.168\.0\.2 SOURCE_KEY = MetaData:Host DEFAULT_VALUE = as400 DEST_KEY = MetaData:Sourcetype

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf

The other, probably easier way, is to use a syslog-ng server, use specific destinations for each ip, and then index the resulting output. http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/overview

Re: spliting multiple feed that use udp:514

jarjoh42 — Fri, 21 Dec 2012 20:15:53 GMT

I used your template changed the names of the stanzas to match what the default valuse are for each of the apps as you can see.

[cisco_asa] REGEX = 192\.20\.9\.9|192\.20\.9\.10 SOURCE_KEY = MetaData:Host DEFAULT_VALUE = cisco_asa DEST_KEY = MetaData:Sourcetype

[dspjrn:5] REGEX = 192\.16\.9\.20 SOURCE_KEY = MetaData:Host DEFAULT_VALUE = as400 DEST_KEY = MetaData:Sourcetype

is this data still indexed to the index_syslog. is there a way to change the index for just the dspjrn:5 source(from props.conf in iseries app)to index_iseries to match the index.conf in the iseries app's index.conf file.

I would leave this as a comment but it will not let me post it as comment

Re: spliting multiple feed that use udp:514

alacercogitatus — Fri, 21 Dec 2012 20:36:15 GMT

You could perform Double Transforms (add a second stanza for each sourcetype and set the _MetaData:Index to the index you want). But to better utilize the processing power, you probably should just collect the syslog onto the server, and index from there.

Re: spliting multiple feed that use udp:514

jarjoh42 — Mon, 28 Sep 2020 14:05:25 GMT

Final solution

I had to do all this in the splunk/system/local. this separated the host out of the data and set them into their own source type. Please note that this is still all in the same indexed. The next goal would to send each sourcetype to its own index. Also when I was doiing the calls in the app I had to use [sourcetype::cisco_asa] for it to accept the parameter.

inputs.conf

[udp://514]<
connection_host = ip
index = index_syslog

props.conf

[source::udp:514]
TRANSFORMS-changesourcetype = cisco_esa_st, cisco_asa_st, as400_st

transforms.conf

[cisco_asa_st]
REGEX = 192.x.x.x|192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_asa
DEST_KEY = MetaData:Sourcetype

[cisco_esa_st]
REGEX = 192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_esa
DEST_KEY = MetaData:Sourcetype

[as400_st]
REGEX = 192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::as400
DEST_KEY = MetaData:Sourcetype