Getting Data In

How do you modify data values before indexing?

soumyacharya91
Path Finder

Hi All,

I want to remove more than 2 white spaces from event values at heavy forwarder before ingesting to indexer. Can anyone guide me with this change so that I can be able to fix the issue.

Current State :

field1="xxxxxx", field2="xxx                          ", field3="xxx    ", field4="x", field5="xxxx                                                                       ", field6="xxx  ", field7="xxx                                                                                            ", field8="xxxx                                                                                           ", field9="xxxxx                                                                                                                                                                                                                                   ", field10="xxxxx"

Required State
field1="xxxxxx", field2="xxx", field3="xxx", field4="x", field5="xxxx", field6="xxx", field7="xxx", field8="xxxx", field9="xxxxx", field10="xxxxx"

0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

View solution in original post

harsmarvania57
SplunkTrust
SplunkTrust

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

soumyacharya91
Path Finder

It is coming from db input.

0 Karma

soumyacharya91
Path Finder

Thanks a lot It is now fixed. You deserve a Chocolate !!

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Great it worked, chcolate pending 🙂

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

You can implement above configuration on server on which Splunk DB Connect is installed, here I am guessing that you have configured DB Input in DB Connect to pull data at certain interval from Database.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...