How do you modify data values before indexing?

[soumyacharya91]

Hi All,

I want to remove more than 2 white spaces from event values at heavy forwarder before ingesting to indexer. Can anyone guide me with this change so that I can be able to fix the issue.

Current State :

field1="xxxxxx", field2="xxx                          ", field3="xxx    ", field4="x", field5="xxxx                                                                       ", field6="xxx  ", field7="xxx                                                                                            ", field8="xxxx                                                                                           ", field9="xxxxx                                                                                                                                                                                                                                   ", field10="xxxxx"

Required State
field1="xxxxxx", field2="xxx", field3="xxx", field4="x", field5="xxxx", field6="xxx", field7="xxx", field8="xxxx", field9="xxxxx", field10="xxxxx"

[harsmarvania57 · Accepted solution]

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

[soumyacharya91]

It is coming from db input.

[soumyacharya91]

Thanks a lot It is now fixed. You deserve a Chocolate !!

[harsmarvania57]

Great it worked, chcolate pending 🙂

[harsmarvania57]

You can implement above configuration on server on which Splunk DB Connect is installed, here I am guessing that you have configured DB Input in DB Connect to pull data at certain interval from Database.