topic Filter csv logs before indexing in Getting Data In

Filter csv logs before indexing

krylov — Wed, 17 May 2017 13:24:21 GMT

I want exclude fields bar and baz with all their values before indexing.

I have CSV log:
foo,bar,baz
abc,123,456
abcd,234,567
abcd,456,567
bcd,345,678

How I can do it?

Thanks in advance!

kmorris_splunk — Wed, 17 May 2017 14:02:07 GMT

Try this method:

I have tested this with a CSV file and it works.

SAMPLE DATA

field1,field2,field3,field4
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4

props.conf

[excludefields_ex]
TRANSFORMS-somefields = somefields

transforms.conf

[somefields]
DEST_KEY = _raw
REGEX = (\S+),(\S+),(\S+),(\S+)
FORMAT =  $1 $3

krylov — Thu, 18 May 2017 10:21:53 GMT

Thanks you! It really works