How to extract the csv fields at index-time?

[rajasekhar14]

hi all,
i'm trying extract the fields from the csv files and my csv file is looks like this,

just want to extract all fields at index-time only.

field1,filed2,-,-,-,etc

and my props.conf is
[sourcetype]
INDEXED_EXTRACTIONS = CSV
FIELD_DELIMITER = ,
HEADER_FIELD_DELIMITER = ,

but this is not successful, am i missing something ?

[ashajambagi]

@rajasekhar14 Where have you placed your props.conf? can you show the stanza in inputs.conf.

Refer this link - https://answers.splunk.com/answers/719666/data-not-getting-extracted-correctly-as-per-csv.html

[rajasekhar14]

@ashajambagi here is the my inputs.conf
[monitor:/D:\mytest/splunk.csv]
sourcetype=test
index=myindex
crcSalt =
initCrcLength = 256

[ashajambagi]

Check the format for this : [monitor:/D:\mytest/splunk.csv]

[sourcetype] #have you mentioned test here instead of sourcetype?
INDEXED_EXTRACTIONS = CSV
FIELD_DELIMITER = ,
HEADER_FIELD_DELIMITER = ,

[woodcock · Accepted solution]

The INDEXED_EXTRACTIONS feature, unlike most index-time-related features, actually happens on the UF. So your props.conf must be sent to your UF and Splunk restarted there.

[biko]

thanks woodcok, this saved my day, at least what was left of it after struggling for hours.

This behaviour seems very counter-intuitive; I am used to the concept of UFs beeing dumb and having no notion of events

[rajasekhar14]

@woodcock, as per all your suggestions i placed these settings in UF and restarted it, but now no luck.

[woodcock]

What do you mean "no luck" exactly?

Is your data coming in to splunk? If so, then it definitely is working.
With INDEXED_EXTRACTIONS it is ALL or NONE.
I suspect that you expect this change to fix data that is already in wrong. It will NOT do that. You have to send NEW data in, and then it should work. If data is not coming in, then the only thing that might be causing you a problem is that your sourcetype does not match or your Timestamping is wrong so the events are ending up in a timeframe that you did not expect. Try a timepicker with All time to check for the latter.

[rajasekhar14]

@woodcock now its working. previously it didn't deployed to UF. i have a question that why we need to deployed to UF only? in my case UF is forwarding to HF, and HF is forwarding to Indexers. So my all parsing is happening in HF level to avoid load on Indexers.

[woodcock]

Parsing on HF swaps CPU load for port I/O load and a different CPU load in that payload per event is much fatter and is very inefficient. See here:

https://www.splunk.com/blog/2016/12/12/universal-or-heavy-that-is-the-question.html

[chrisyounger]

Hi @rajasekhar14

Your config looks correct. Just make sure this props.conf file Is on the universal forwarder and not the indexer.

All the best

[rajasekhar14]

once i deployed these settings to UF its working

[rajasekhar14]

Hi Chris,
I haven’t deployed to UF, because we have HF in place between Indexers and UFs. So I deployed to HF.

[chrisyounger]

Confusingly, CSV indexed extractions actually happen on the universal forwarder. It needs to be done here becuase it needs to use the header of the file regularly so it knows the column names.

[ddrillic]

Absolutely - a bit more at Do we need props.conf on the indexer when indexing a csv file?

[chrisyounger]

No I don't think so. However it doesn't harm to have it there.

[rajasekhar14]

Chris, we are parsing at HF level, so I deployed tob HF.