topic Re: how to separate data from different sources going to same syslog port? in Getting Data In

how to separate data from different sources going to same syslog port?

fernandoandre — Thu, 09 Feb 2012 18:14:16 GMT

Hi, I need help with the following.

I have two distinct services in different machines. I want to send their logs/information to Splunk (in another machine/server). The two services can only send logs through udp/514 syslog. I can't change this setting at those services and I don't have where to write logs locally.

How can I receive data in Splunk in the same port and separate data coming from different services to different indexes?

In inputs.conf for example I can't add more indexes right? And to filter the data to one index or another I need transforms.conf and props.conf! How?

inputs.conf (at Splunk server)

[udp://514]

connection_host = ip

index = my_syslog1

sourcetype = syslog1

Re: how to separate data from different sources going to same syslog port?

clintsharp — Thu, 09 Feb 2012 18:41:19 GMT

Make transform.conf entries which will rewrite the index:

[Host1IndexRewrite]
DEST_KEY = _MetaData:Index
FORMAT = <Host1Index>

[Host2IndexRewrite]
DEST_KEY = _MetaData:Index
FORMAT = <Host2Index>

And in props.conf to match on the incoming host:

[host::<host1>]
TRANSFORMS-host1indexrewrite = Host1IndexRewrite

[host::<host2>]
TRANSFORMS-host2indexrewrite = Host2IndexRewrite

Now, one caveat, this would put everything from those hosts in that index, not just based on source. Hope that helps!

Edit: Someone pointed out to me you may need a regex to match before you can use format. If the first doesn't work, the following would be worth a try for transforms.conf:

[Host1IndexRewrite]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = <Host1Index>

[Host2IndexRewrite]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = <Host2Index>

Re: how to separate data from different sources going to same syslog port?

fernandoandre — Mon, 28 Sep 2020 11:22:28 GMT

sorry, this is unclear for me! Could you please clarify?

1) I leave everything like it is now on inputs.conf??

2) what is this supposed to do "_MetaData:Index"?

3) I would expect to see something like index=my_index1 and index=my_index2 for redirecting different sources. Will "Format" accomplish the same thing?

Re: how to separate data from different sources going to same syslog port?

clintsharp — Thu, 09 Feb 2012 19:54:52 GMT

1) Yes, leave everything as specified in inputs.conf. Essentially the index you specify becomes the default index. In fact, you could simply make one props.conf and transforms.conf entry to override the index from the second host if you like.

2) That changes the the index that the input writes to at index time.

3) Format is the format for how to write to the field. You're overriding the index in the transform. Format can be used with regexes as well (see http://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf), but since we're not we can just specify the index name.