Getting Data In

Filter csv logs before indexing

krylov
Explorer

I want exclude fields bar and baz with all their values before indexing.

I have CSV log:
foo,bar,baz
abc,123,456
abcd,234,567
abcd,456,567
bcd,345,678

How I can do it?

Thanks in advance!

Tags (3)
0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

Try this method:

https://answers.splunk.com/answers/109253/how-to-filter-or-extract-fields-before-indexing-time.html

I have tested this with a CSV file and it works.

SAMPLE DATA

field1,field2,field3,field4
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4

props.conf

[excludefields_ex]
TRANSFORMS-somefields = somefields

transforms.conf

[somefields]
DEST_KEY = _raw
REGEX = (\S+),(\S+),(\S+),(\S+)
FORMAT =  $1 $3

krylov
Explorer

Thanks you! It really works

0 Karma
Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...