topic How do you modify data values before indexing? in Getting Data In

How do you modify data values before indexing?

soumyacharya91 — Thu, 06 Sep 2018 12:03:24 GMT

Hi All,

I want to remove more than 2 white spaces from event values at heavy forwarder before ingesting to indexer. Can anyone guide me with this change so that I can be able to fix the issue.

Current State :

field1="xxxxxx", field2="xxx                          ", field3="xxx    ", field4="x", field5="xxxx                                                                       ", field6="xxx  ", field7="xxx                                                                                            ", field8="xxxx                                                                                           ", field9="xxxxx                                                                                                                                                                                                                                   ", field10="xxxxx"

Required State
field1="xxxxxx", field2="xxx", field3="xxx", field4="x", field5="xxxx", field6="xxx", field7="xxx", field8="xxxx", field9="xxxxx", field10="xxxxx"

Re: How do you modify data values before indexing?

harsmarvania57 — Thu, 06 Sep 2018 12:59:49 GMT

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

Re: How do you modify data values before indexing?

soumyacharya91 — Thu, 06 Sep 2018 13:27:32 GMT

It is coming from db input.

Re: How do you modify data values before indexing?

harsmarvania57 — Thu, 06 Sep 2018 13:30:52 GMT

You can implement above configuration on server on which Splunk DB Connect is installed, here I am guessing that you have configured DB Input in DB Connect to pull data at certain interval from Database.

Re: How do you modify data values before indexing?

soumyacharya91 — Thu, 06 Sep 2018 13:37:18 GMT

Thanks a lot It is now fixed. You deserve a Chocolate !!

Re: How do you modify data values before indexing?

harsmarvania57 — Thu, 06 Sep 2018 13:39:23 GMT

Great it worked, chcolate pending 🙂