Getting Data In

Why is my SEDCMD props.conf file not working?

Explorer

I have created a props.conf file under etc/system/local/props.conf

The content is

[default]
SEDCMD-ipi2 = y/e/g/g

I would expect that any letter e would be replaced with a g but it's not working.

Should I activate in any way this file or something like that?

Tags (2)
1 Solution

SplunkTrust
SplunkTrust

If you just dropped the file there then you need to restart Splunk since that's an index-time setting.

Once you've done that you can use this call to confirm Splunk reading that file:

./bin/splunk cmd btool --debug props list

View solution in original post

SplunkTrust
SplunkTrust

Use this

[default]
SEDCMD-ipi2 = s/e/g/g

option "y" is for strings and does't do global replacement.

0 Karma

Explorer

Even this does not work.

0 Karma

SplunkTrust
SplunkTrust

If you just dropped the file there then you need to restart Splunk since that's an index-time setting.

Once you've done that you can use this call to confirm Splunk reading that file:

./bin/splunk cmd btool --debug props list

View solution in original post

SplunkTrust
SplunkTrust

For reference here's a quick comparison: http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Typesofforwarders#Forwarder_comparison

"Per-Event filtering" is what you're doing here, basically.

Builder

I have added SEDCMD on heavy forwarder and it does not work. I have tested the settings individually using UI and it works.

0 Karma

SplunkTrust
SplunkTrust

You can do that, just not on a universal forwarder. Replace it with a heavy forwarder that does all the parsing, timestamping, regexreplacement, yada yada on the source machine and you're there. Then you will need this props.conf setting on the heavy forwarder.

0 Karma

Explorer

Ok now I understand what universal forwarder client means. Thanks 🙂

0 Karma

Explorer

From my point of view it makes more sense to anonymize data from the Forwarder and not on the server. That's why I always tried to configure the forwarder.

0 Karma

SplunkTrust
SplunkTrust

Yeah, this does belong on the indexer if you're sending it from a universal forwarder. Some newer settings also require some props.conf keys to be present on the forwarder, but stuff like SEDCMD doesn't - regardless of the version.

0 Karma

Explorer

I found it! I was configuring the forwarder and not the server! Note with 6.1 props.conf might have to go on the UFs. From dev: With 6.1, the structured data props.conf are happening at monitoring time therefore the props.conf has also to be on the forwarders. makes me on the right way. Thanks for the help 🙂

Motivator

Are you saying that props settings for structured data on the forwarder were interfering with props.conf settings on the indexer because the "monitoring time" for the inputs needs structured settings, but structured settings end the data parsing phase so that it cannot happen on the indexers?

0 Karma

Explorer

I even try with another server, it does not work... Our forwarder has the version 5.0.4.

0 Karma

SplunkTrust
SplunkTrust

Well, something's fishy in your Splunk environment... I've just created a props.conf in system/local like this:

[default]
SEDCMD-ipi2 = y/e/g/

and now get events like this:

127.0.0.1 - admin [31/Oct/2014:22:02:08.023 +0100] "GET /sgrvicgsNS/admin/sgarch/sgarch/typgahgad?prgfix=indgx%3D_intgrnal&count=50&max_timg=1&output_modg=json HTTP/1.0" 200 73 - - - 2ms

Looks fing to mg.

Explorer

Yes, i'm refreshing the page, which produce a log with the date, so I'm sure.

0 Karma

SplunkTrust
SplunkTrust

You are looking at stuff indexed after setting this, right?

0 Karma

Explorer

Even with y/e/g the props.conf does not work...

0 Karma

SplunkTrust
SplunkTrust

The way I understand it, y/e/g/ shouldn't need the g flag because character substitution only makes sense globally... do try leaving off the global flag.

0 Karma

Explorer

I always restarted before. With your debug option I even can read /opt/splunkforwarder/etc/system/local/props.conf SEDCMD-ipi2 = y/e/g/g. It seams that it does not appear as much as the other lines. But the letter e still appears on the splunk server.

0 Karma