topic Re: Replace spaces not working in props.conf SEDCMD in Splunk Search

Replace spaces not working in props.conf SEDCMD

jlvix1 — Wed, 17 Aug 2016 11:01:15 GMT

Hi all, I have tried everything...

In props.conf

SEDCMD-replacespaces = s/\s/_/g

and also

SEDCMD-replacespaces = s/ /_/g

I tried something like s/\s/lol/g and this did not work also, the problem is likely with identifying the single space.

What I would like to do is perform a find replace for spaces and replace with underscores, as the KV pairs are not being picked up correctly when a space is encountered, for example I have the code in (that works) that performs : SEDCMD-replacecolons = s/\s:\s/=/g s/:\s/=/g s/:\n/=/g t hat puts the equals signs in correctly, I just want to re-go over it and replace spaces with underscores.

We aren't able to change the code that writes the logs, in case anyone suggests that.

Can anyone please explain why sedcmd will not work with a single space as either a regex identifier or a raw character?

Re: Replace spaces not working in props.conf SEDCMD

somesoni2 — Thu, 18 Aug 2016 17:07:49 GMT

I don't see an issue with the SEDCMD command. Where are you placing this props.conf? This should go on Indexer/Heavy Forwarder and will only do the replacement for new events (old events will not change).

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Fri, 19 Aug 2016 07:40:08 GMT

Yes this is in props.conf. As I stated, replacecolons works fine. I have re-tried various times on various configurations. The problem is replacing \s on its own. Is it working for you?

Re: Replace spaces not working in props.conf SEDCMD

Raghav2384 — Fri, 19 Aug 2016 12:04:04 GMT

Can you try s/\ /_/g?

That has done trick once for me.

Thanks,
Raghav

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Fri, 19 Aug 2016 12:56:19 GMT

Hi, this does not work, still not bringing in the events, SEDCMD has an issue with this

Re: Replace spaces not working in props.conf SEDCMD

jkat54 — Fri, 19 Aug 2016 14:33:26 GMT

how about \s+ or putting the sedcmd in quotes?

Are there spaces when the sedcmd runs?

example

raw = THERE ARE SPACES
SEDCMD-AAA = s/THERE\sARE\sSPACES/TEHREARENTSPACES/g
SEDCMD-BBB = s/\s//g

Re: Replace spaces not working in props.conf SEDCMD

jkat54 — Fri, 19 Aug 2016 14:33:47 GMT

Are you restarting splunk? Are you doing this on indexers and forwarders?

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Fri, 19 Aug 2016 15:14:37 GMT

Hi, yes everything is working otherwise, doing this on heavy forwarder.

I'm looking at other options now, going to use regex capture groups and just try to scrape what I can.

My testimony here is that SED in splunk does not work identically to SED in linux. \s can not be found and replaced with anything in splunk. When reading up on SED in general, my command should work fine.

This may of course be a collision between transforms.conf etc... Or it could just be that there is some method required.

Re: Replace spaces not working in props.conf SEDCMD

Raghav2384 — Fri, 19 Aug 2016 15:29:10 GMT

I just tried the following from search, which is pretty much a mimic in props without the """

|gentimes start=-1|eval Space="THIS  IS   A   TEST"|rex mode=sed field=Space "s/( )/^/g" and the output is 

THIS^^IS^^^A^^^TEST

One last try may be?

Thanks,
Raghav

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Fri, 19 Aug 2016 15:36:52 GMT

Hi, I'd imagine that will work here as well, though the search time stuff is a fairly different game to the indexing side?

Re: Replace spaces not working in props.conf SEDCMD

jkat54 — Fri, 19 Aug 2016 16:02:33 GMT

 \s works fine in sedcmd for me

Re: Replace spaces not working in props.conf SEDCMD

Raghav2384 — Fri, 19 Aug 2016 16:16:53 GMT

+1 to @jkat54 's answer.

I created a script that writes "$Date T O D A Y I S $DAY" and configured the monitored input.

Inputs.conf
[monitor:///opt/splunk/bin/scripts/test.txt]
index=main
sourcetype=test

In props.conf,

[source::/opt/splunk/bin/scripts/test.txt]
SEDCMD-replacespaces = s/()/^/g

Bounced splunkd and here's the output in search

index=main sourcetype=test

Output:

^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^M^O^N^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^T^U^E^S^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^W^E^D^N^E^S^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^T^H^U^R^S^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^F^R^I^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^S^A^T^U^R^D^A^Y^
^0^8^-^1^9^-^2^0^1^6^ ^T^O^D^A^Y^ ^I^S^ ^S^U^N^D^A^Y

I ran out of suggestions as all the proposed formats work for me.

Hope this helps!

Thanks,
Raghav

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Mon, 22 Aug 2016 07:39:08 GMT

Thanks for that, slightly different to \s:

SEDCMD-replacespaces = s/()/^/g

I will give that a shot!

Cheers

Re: Replace spaces not working in props.conf SEDCMD

jkat54 — Mon, 22 Aug 2016 15:58:40 GMT

We're going to have to move comments here because we've reached the limit on comments under one question.

Real quick @jlvix1, is this windows or what?

Re: Replace spaces not working in props.conf SEDCMD

jlvix1 — Tue, 23 Aug 2016 07:42:15 GMT

Yes windows, 7 and 2012 I have tried this on. It's just a bit hacky that SEDCMD-replacespaces = s/()/^/g means to replace any character with ^ and including spaces.