Force inclusion of space character as a first character in FORMAT string in transforms.conf

[wowbaggerHU]

Hello everyone!

I am experimenting with the SC4S transforms that are posted here:
https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/

My problem is that I am trying to reformat fields, and in one particular place I would need to ensure that a space preceeds the _h= part in the transform stanza below.

[md_host]
SOURCE_KEY = MetaData:Host
REGEX = ^host::(.*)$
FORMAT = _h=$1 $0
DEST_KEY = _raw

However if I add multiple whitespaces in the FORMAT string, right after the equals sign in the above example, they will be ignored.
Should put the whole thing betweem quotes? Wouldn't the quotes be included in the _raw string?

What would be the right solution for this?

[wowbaggerHU]

@tscrogginsThanks for the idea. It worked, though I added my own set of modifications to it.

As a final touch I would like to put the relevant part of my config here, so as to contribute it back to the community:

[md_host]
INGEST_EVAL = _raw=" _h=".host." "._raw

[md_subsecond]
SOURCE_KEY = _meta
REGEX = _subsecond=(\.\d+)
FORMAT = $1$0
DEST_KEY = _raw

[md_time]
SOURCE_KEY = _time
REGEX = (.*)
FORMAT = _ts=$1$0
DEST_KEY = _raw

[tscroggins]

Looks good! Just be mindful of the difference between the plus (+) operator and the dot (.) operator. Plus concatenates strings and adds numbers, but dot concatenates both strings and numbers as strings. If you're unsure of the order of operations or the variable value in other contexts, you can wrap the variables in the tostring() function.

It's interesting that Splunk also assumed a leading space would work in the SC4S transforms:

[metadata_time]
SOURCE_KEY = _time
REGEX = (.*)
FORMAT =  t="$1$0
DEST_KEY = _raw

@PickleRick's suggestion to escape the space with i.e. a blackslash just evaluates to a literal backslash followed by a space.

[wowbaggerHU]

Thanks for the explanation, it was educational. I have corrected the plus operators where appropriate.
On the other hand, I have now triple-checked, and in deed, multiple leading whitespaces are ignored in the FORMAT string.

But yes, it would seem that Splunk or whoever wrote the SC4S config assumed that they would be honored.

[PickleRick]

Apart from @tscroggins 's solution you could try escaping your initial space. It should show the config parser that there is a non-space character so your key-value pair in config is split properly but since the space doesn't normally need escaping it shouldn't hurt.

[wowbaggerHU]

Good idea, I tried it, but unfortunately it doesn't seem to work. I have this configured:

[md_host]
SOURCE_KEY = MetaData:Host
REGEX = ^host::(.*)$
FORMAT = \ _h=$1 $0
DEST_KEY = _raw

[md_subsecond_default]
SOURCE_KEY = _meta
REGEX = _subsecond=(\.\d+)
FORMAT = $1$0
DEST_KEY = _raw

[md_time_default]
SOURCE_KEY = _time
REGEX = (.*)
FORMAT = _ts=$1$0
DEST_KEY = _raw

And I get this:

        0x0040:  e073 339e e073 339e 3232 3738 205f 7473  .s3..s3.2278._ts
        0x0050:  3d31 3733 3732 3739 3038 315c 205f 683d  =1737279081\._h=
        0x0060:  7370 6c75 6e6b 2d68 6620 5f69 6478 3d5f  splunk-hf._idx=_

 But I agree, this would have been the most elegant solution.

[tscroggins · Accepted solution]

Hi @wowbaggerHU,

You can use INGEST_EVAL as a workaround:

# transforms.conf
[md_host]
INGEST_EVAL = _raw:=" h=\"".host."\" "._raw