Solved: Re: Using transforms.conf to change metadata forma... - Page 2

wowbaggerHU · ‎01-12-2025

Hello everyone!

I would like to ask about the Splunk Heavy Forwarder Splunk-side config:
https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/

With those settings it will send the metadata in the format of key::value.
Is it possible to reconfigure it to send metadata key-value pairs with some other key-value separator instead of "::"?
If yes, how exactly?

PickleRick · ‎01-13-2025

OK. So this is not about Splunk's metadata format as much as rendering it for export.

I suppose you can tweak it a little.

The key part here is this transform

[metadata_meta]
SOURCE_KEY = _meta
REGEX = (?ims)(.*)
FORMAT = ~~~SM~~~$1~~~EM~~~$0 
DEST_KEY = _raw

It's being called as the first one (except for the one manipulating routing) and it exports whole _meta as-is.

So you need to change it to:

[sanitize_metadata]
INGEST_EVAL = escaped_meta=replace(_meta,"::","=")
[metadata_meta]
SOURCE_KEY = escaped_meta
REGEX = (?ims)(.*)
FORMAT = ~~~SM~~~$1~~~EM~~~$0 
DEST_KEY = _raw

And of course adjust props to call the sanitize_metadata first

TRANSFORMS-zza-syslog = syslog_canforward, sanitize_metadata, metadata_meta,  metadata_source, metadata_sourcetype, metadata_index, metadata_host, metadata_subsecond, metadata_time, syslog_prefix, syslog_drop_zero

View solution in original post

wowbaggerHU · ‎01-13-2025

Hello,

I have a Windows machine with UF that sends its logs to a HF, which has the SC4S derived config loaded (see the opening entry's link). That allows to reformat the logs that passed through the HF to IETF 5424 syslog (with framing enabled) and forward them to a syslog instance.

That reformatting pretty much alters most parts of the original message.
In the output you will generally see the first half of the message (not counting the SDATA part) will contain the metadata fields in the key::value format.

I would like to change that in the syslog output generated by the config on the HF node.

PickleRick · ‎01-13-2025

As a side note - pretty much every solution involving Windows and third party syslog breaks stuff somewhere.

Either breaks Splunk parsing or breaks the third party parsing. At some point something is almost sure to break.

Using transforms.conf to change metadata format from key::value

administration

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Announcing Modern Navigation: A New Era of Splunk User Experience

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

Join the Conversation