Getting Data In

Filter csv logs before indexing

krylov
Explorer

I want exclude fields bar and baz with all their values before indexing.

I have CSV log:
foo,bar,baz
abc,123,456
abcd,234,567
abcd,456,567
bcd,345,678

How I can do it?

Thanks in advance!

Tags (3)
0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

Try this method:

https://answers.splunk.com/answers/109253/how-to-filter-or-extract-fields-before-indexing-time.html

I have tested this with a CSV file and it works.

SAMPLE DATA

field1,field2,field3,field4
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4

props.conf

[excludefields_ex]
TRANSFORMS-somefields = somefields

transforms.conf

[somefields]
DEST_KEY = _raw
REGEX = (\S+),(\S+),(\S+),(\S+)
FORMAT =  $1 $3

krylov
Explorer

Thanks you! It really works

0 Karma
Get Updates on the Splunk Community!

Your Guide to Splunk Digital Experience Monitoring

A flawless digital experience isn't just an advantage, it's key to customer loyalty and business success. But ...

Data Management Digest – November 2025

  Welcome to the inaugural edition of Data Management Digest! As your trusted partner in data innovation, the ...

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...