Getting Data In

If I have 70+ devices writing data to UDP port 514, how do I route 5 IP addresses to one index, and the rest to another index?

allan_newton
Path Finder

Hi,

I have a very specific problem. I have more than 70 devices writing data to UDP port 514. Now I need to input five ip_addresses to one index and the rest of them to another index.

eg: 10.0.0.1, 10.0.0.2, 10.0.0.3, 10.0.0.4, 10.0.0.5, ...... 10.0.0.70

Now 10.0.0.1, 10.0.0.2... 10.0.0.5 should go to the index=indx1 and sourcetype=srctype1

and the rest of the IP addresses should go to another index.

Note: Unlike the above example ip_addresses in my case are not incremental or doesn't have any pattern.

Please help.

0 Karma
1 Solution

dshpritz
SplunkTrust
SplunkTrust

Here is a simple way to do this using props and transforms:

For each host you would need a props.conf entry that calls a transform

[host::10.0.0.1]
TRANSFORMS-force_index = force_index, force_sourcetype

Then in your transforms.conf:

[force_index]
REGEX = (.)
DEST_KEY = _MetaData:Index
FORMAT = indx1

[force_sourcetype]
REGEX = (.)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::srctype1

These configurations would need to be placed on your first parsing system (first full instance of Splunk, like a heavy forwarder or indexers).

HTH,

Dave

View solution in original post

dshpritz
SplunkTrust
SplunkTrust

Here is a simple way to do this using props and transforms:

For each host you would need a props.conf entry that calls a transform

[host::10.0.0.1]
TRANSFORMS-force_index = force_index, force_sourcetype

Then in your transforms.conf:

[force_index]
REGEX = (.)
DEST_KEY = _MetaData:Index
FORMAT = indx1

[force_sourcetype]
REGEX = (.)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::srctype1

These configurations would need to be placed on your first parsing system (first full instance of Splunk, like a heavy forwarder or indexers).

HTH,

Dave

allan_newton
Path Finder

@dshpritz Thank you very much.

There is a different index in inputs.conf for the port. Still I can force it to take up a new index in transforms.conf ??

Thanks.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The transforms will overwrite what was set in inputs.conf, yes.

alacercogitatus
SplunkTrust
SplunkTrust

Also: If you like the syslog-ng route, see this blog that lays it out in best practices: http://www.georgestarcher.com/splunk-success-with-syslog/

allan_newton
Path Finder

If I'm doing it on the forwarder. Can I do it in the etc/system/default folder's transforms.conf & props.conf?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You do NOT want to change things in .../default unless you're the author of that app - and you're not the author of system at all. For example, any splunk upgrade would overwrite your changes.

system/local would work, or apps/your_app/default to bundle that setting in a neat deployable package.

0 Karma

wrangler2x
Motivator

Unless he is doing it via deployment server in $SPLUNK_HOME/etc/deployment-apps/deployment_app_name/default

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Then he'd be the author of that app, yes.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...