Why is my SEDCMD props.conf file not working?

[cbou]

I have created a props.conf file under etc/system/local/props.conf

The content is

[default]
SEDCMD-ipi2 = y/e/g/g

I would expect that any letter e would be replaced with a g but it's not working.

Should I activate in any way this file or something like that?

[somesoni2]

Use this

[default]
SEDCMD-ipi2 = s/e/g/g

option "y" is for strings and does't do global replacement.

[cbou]

Even this does not work.

[martin_mueller · Accepted solution]

If you just dropped the file there then you need to restart Splunk since that's an index-time setting.

Once you've done that you can use this call to confirm Splunk reading that file:

./bin/splunk cmd btool --debug props list

[martin_mueller]

For reference here's a quick comparison: http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Typesofforwarders#Forwarder_comparison

"Per-Event filtering" is what you're doing here, basically.

[nawazns5038]

I have added SEDCMD on heavy forwarder and it does not work. I have tested the settings individually using UI and it works.

[martin_mueller]

You can do that, just not on a universal forwarder. Replace it with a heavy forwarder that does all the parsing, timestamping, regexreplacement, yada yada on the source machine and you're there. Then you will need this props.conf setting on the heavy forwarder.

[cbou]

Ok now I understand what universal forwarder client means. Thanks 🙂

[cbou]

From my point of view it makes more sense to anonymize data from the Forwarder and not on the server. That's why I always tried to configure the forwarder.

[martin_mueller]

Yeah, this does belong on the indexer if you're sending it from a universal forwarder. Some newer settings also require some props.conf keys to be present on the forwarder, but stuff like SEDCMD doesn't - regardless of the version.

[cbou]

I found it! I was configuring the forwarder and not the server! Note with 6.1 props.conf might have to go on the UFs. From dev: With 6.1, the structured data props.conf are happening at monitoring time therefore the props.conf has also to be on the forwarders. makes me on the right way. Thanks for the help 🙂

[landen99]

Are you saying that props settings for structured data on the forwarder were interfering with props.conf settings on the indexer because the "monitoring time" for the inputs needs structured settings, but structured settings end the data parsing phase so that it cannot happen on the indexers?

[cbou]

I even try with another server, it does not work... Our forwarder has the version 5.0.4.

[martin_mueller]

Well, something's fishy in your Splunk environment... I've just created a props.conf in system/local like this:

[default]
SEDCMD-ipi2 = y/e/g/

and now get events like this:

127.0.0.1 - admin [31/Oct/2014:22:02:08.023 +0100] "GET /sgrvicgsNS/admin/sgarch/sgarch/typgahgad?prgfix=indgx%3D_intgrnal&count=50&max_timg=1&output_modg=json HTTP/1.0" 200 73 - - - 2ms

Looks fing to mg.

[cbou]

Yes, i'm refreshing the page, which produce a log with the date, so I'm sure.

[martin_mueller]

You are looking at stuff indexed after setting this, right?

[cbou]

Even with y/e/g the props.conf does not work...

[martin_mueller]

The way I understand it, y/e/g/ shouldn't need the g flag because character substitution only makes sense globally... do try leaving off the global flag.

[cbou]

I always restarted before. With your debug option I even can read /opt/splunkforwarder/etc/system/local/props.conf SEDCMD-ipi2 = y/e/g/g. It seams that it does not appear as much as the other lines. But the letter e still appears on the splunk server.