Getting Data In

Filter csv logs before indexing

krylov
Explorer

I want exclude fields bar and baz with all their values before indexing.

I have CSV log:
foo,bar,baz
abc,123,456
abcd,234,567
abcd,456,567
bcd,345,678

How I can do it?

Thanks in advance!

Tags (3)
0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

Try this method:

https://answers.splunk.com/answers/109253/how-to-filter-or-extract-fields-before-indexing-time.html

I have tested this with a CSV file and it works.

SAMPLE DATA

field1,field2,field3,field4
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4

props.conf

[excludefields_ex]
TRANSFORMS-somefields = somefields

transforms.conf

[somefields]
DEST_KEY = _raw
REGEX = (\S+),(\S+),(\S+),(\S+)
FORMAT =  $1 $3

krylov
Explorer

Thanks you! It really works

0 Karma
Get Updates on the Splunk Community!

App Platform's 2025 Year in Review: A Year of Innovation, Growth, and Community

As we step into 2026, it’s the perfect moment to reflect on what an extraordinary year 2025 was for the Splunk ...

Operationalizing Entity Risk Score with Enterprise Security 8.3+

Overview Enterprise Security 8.3 introduces a powerful new feature called “Entity Risk Scoring” (ERS) for ...

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...