topic Re: Filter columns from CSV while monitoring in Getting Data In

Filter columns from CSV while monitoring

ankitarath2011 — Thu, 22 Aug 2019 09:31:03 GMT

I am monitoring a folder with csv files with 400+ fields, out of which need only 50 fields for my dashboard.
Can we do something in Indexing time, so that it will not index the other 350 fields which are not required.

Removing from CSV is not possible. So, need to handle it in splunk only.

Please suggest.

Re: Filter columns from CSV while monitoring

KailA — Thu, 22 Aug 2019 14:09:18 GMT

Hello,

I don't know if it's working for 400+ fields but I found this post for you : https://answers.splunk.com/answers/529138/filter-csv-logs-before-indexing.html

Re: Filter columns from CSV while monitoring

ankitarath2011 — Fri, 23 Aug 2019 05:32:55 GMT

Ok, I will try this

Re: Filter columns from CSV while monitoring

KailA — Fri, 23 Aug 2019 07:20:38 GMT

Let me know if it works

Re: Filter columns from CSV while monitoring

woodcock — Mon, 02 Sep 2019 01:51:35 GMT

You can do this with a carefully constructed SEDCMD setting but this may not work if you are using INDEXED_EXTRACTIONS=csv (then again, it may very well work). I know that it definitely will work if you are not using INDEXED_EXTRACTIONS.