Getting Data In

Why are props and transforms preconfigured to sanitize the hostname for pretrained sourcetypes like 'syslog' and 'linux_syslog_messages'?

HIBE151
Explorer

Hello,

Just a simple question about pretrained sourcetypes like 'syslog' and 'linux_syslog_messages'.
Why are those props and transforms preconfigured to sanitize the hostname?

For example, when I have a system with the hostname: "blub.local.com" it will strip this hostname to "blub".
Actually I've heard that Splunk doesn't suggest to change the hostname in a props/transforms if it isn't necessary .
You can still change it in your inputs which doesn't affect your performance that much.
Furthermore when you want to 'splunk' some other logs they will have your real name "blub.local.com".

I can change it manually from /opt/splunk/etc/system/default/ but that is actually just a workaround until the next update.

Does anyone have an idea how to handle this very easily?

Thanks and regards

0 Karma

woodcock
Esteemed Legend

I have the same concern as @DalJeanis but different advice. I would never use $SPLUNK_HOME/etc/system/* at all. Instead, create your own app and deploy it to something like $SPLUNK_HOME/etc/apps/MySyslogApp/default/ and then do the kind of thing that @MicahKemp sugested.

0 Karma

DalJeanis
Legend

You don't want to ever change the .conf files in ...

$SPLUNK_HOME/etc/system/default/ 

...for the reason you just said - that the change will go away, but also for the reason that that is the place you go to get a copy of the default stuff that should always work, if you manage to screw up the configuration files you put anywhere else.

Instead, make your changes in...

$SPLUNK_HOME/etc/system/local/

... but if you are doing a distributed or clustered deployment, then ask for more advice, because the concept of "where" becomes a bit fuzzier.

0 Karma

micahkemp
Champion

The transform in question grabs the hostname from the syslog message itself.

Default props.conf:

[linux_messages_syslog]
pulldown_type = 1
MAX_TIMESTAMP_LOOKAHEAD = 32
TIME_FORMAT = %b %d %H:%M:%S
TRANSFORMS = syslog-host
REPORT-syslog = syslog-extractions
SHOULD_LINEMERGE = False
category = Operating System
description = Format found within the Linux log file /var/log/messages

And the related transforms.conf:

[syslog-host]
DEST_KEY = MetaData:Host
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1

I'm not sure I can fully answer the "why", but I can state the hostname is grabbed from the syslog message, as the above configurations show.

To disable this transform, you can place this in etc/system/local/props.conf:

[linux_messages_syslog]
TRANSFORMS = 
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...