Getting Data In

Original submitting IP for syslog (514/udp) - lost or just hard to find?

gowen
Path Finder

We have multiple F5 appliances submitting their LTM logs via syslog (514/udp). The logs always have the following line format, with "tmm" or "tmm1" being a generic F5 thing rather than the actual hostname:

Oct 19 15:07:47 tmm1 info tmm1[27478]: ...

Splunk is extracting that first 'tmm1' as the hostname and setting host=tmm1. The problem is, since we have multiple F5s submitting logs, they're all grouped as host=tmm1.

1) I don't see the original IP that submitted the syslog entry stored anywhere, which amazes me, because it implies someone could spoof messages and get them rewritten to belong to another host quite easily. Please tell me where I'm missing the original submitting IP?

2) If I can figure out the original submitting IP*, I can use props and transforms to set the host, but is there a way to simply tell Splunk that the submitting IP is the host and not to try and suss it out from the log entries?

  • without knowing the answer to #1, I don't actually know the original submitting IP at this time. The F5 has a dozen IP addresses associated with it, and I don't know which one it is using to send the logs over - it is not, of course, the "main" address that the web GUI sits on.
Tags (2)
0 Karma

jgedeon120
Contributor

Instead of using Splunk to listen to the port set up syslog-ng to receive the messages and then splunk read the log directory. You can have syslog-ng write to a location like /var/log/F5/$HOST/syslog-ng.log then define in props.conf host_segment = 4. The added benefit to this is when you need to restart Splunk you aren't missing any logs.

There is plenty of documentation on setting this up and will also find that it is the recommended way to receive syslog to be processed by Splunk.

kristian_kolb
Ultra Champion

Very good answer from sowings - just wanted to clarify that these transforms that are mentioned, are most likely applied automatically. If your sourcetype is 'syslog' that is definitely the case. To avoid this, change the sourcetype name to something different, e.g. 'syslog-f5'.

If needed set up a separate listening port on the splunk server, specifically for the F5 traffic.

If you wish, have a look in /opt/splunk/etc/system/default/props.conf and transforms.conf to see if there are any transformations occuring for your data.

Hope this helps,

Kristian

0 Karma

gowen
Path Finder

Can I enter multiple REGEX lines in one stanza, or do I need to adjust the existing line to have a REGEX 'THIS' or 'THAT'? Current entry is "REGEX = ^host::1.2.3.4$", what would be the correct OR, "REGEX = ^host::1.2.3.4|host::1.2.3.5$"?

0 Karma

gowen
Path Finder

This seems promising. At least one of our F5s is transformed to SourceType "f5", and the host IP is retained for log messages from that host. I will rewrite this additional F5 to that SourceType and check back in.

0 Karma

sowings
Splunk Employee
Splunk Employee

You may find that a transform called syslog-host or syslog-host-null is being applied to events arriving on UDP/514. This transform remaps the host field based upon the contents of a syslog-type line, which contains the hostname in the event itself. You can check the connection_host attribute in inputs.conf to specify that the source IP should be kept as the host value. You'll also want to make sure that the syslog hostname transforms listed above aren't being applied to your sourcetype.

gowen
Path Finder

There is no such transform - there are syslog transforms that adjust SourceType based on source address, but none adjusting Host. There is no setting for connection_host in inputs.conf, so it should use the default of 'ip'

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...