Getting Data In

How do you modify data values before indexing?

soumyacharya91
Path Finder

Hi All,

I want to remove more than 2 white spaces from event values at heavy forwarder before ingesting to indexer. Can anyone guide me with this change so that I can be able to fix the issue.

Current State :

field1="xxxxxx", field2="xxx                          ", field3="xxx    ", field4="x", field5="xxxx                                                                       ", field6="xxx  ", field7="xxx                                                                                            ", field8="xxxx                                                                                           ", field9="xxxxx                                                                                                                                                                                                                                   ", field10="xxxxx"

Required State
field1="xxxxxx", field2="xxx", field3="xxx", field4="x", field5="xxxx", field6="xxx", field7="xxx", field8="xxxx", field9="xxxxx", field10="xxxxx"

0 Karma
1 Solution

harsmarvania57
Ultra Champion

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

View solution in original post

harsmarvania57
Ultra Champion

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

soumyacharya91
Path Finder

It is coming from db input.

0 Karma

soumyacharya91
Path Finder

Thanks a lot It is now fixed. You deserve a Chocolate !!

0 Karma

harsmarvania57
Ultra Champion

Great it worked, chcolate pending 🙂

0 Karma

harsmarvania57
Ultra Champion

You can implement above configuration on server on which Splunk DB Connect is installed, here I am guessing that you have configured DB Input in DB Connect to pull data at certain interval from Database.

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...