How to configure the REST API Modular Input to parse and extract CSV header and timestamp fields?

[reggie_123]

Hi,

I have configured the REST API Modular Input to receive CSV data using the default handler and having "response_type = text" in inputs.conf.
Now I am trying to make Splunk identify the header fields and the timestamp fields.
I tried to configure the rest input as an indexed CSV extraction in props.conf, and to use timestamp fields, but this did not work, and I concluded that the REST application extractions are somehow not processed at index-time, but rather at search-time.

Is this correct? If so, how do I handle timestamp extraction based on one of the fields and how do I make Splunk parse the field names automatically?

Thanks a lot.

[ryanoconnor]

What REST API are you trying to query? Are you sure the response type is CSV? Can you send a sample response?

Typically if Splunk is monitoring a CSV file there are all types of configurations you can use to parse that data in props.conf http://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf

The following settings seem like they might help you out if you are in-fact monitoring CSV Data.

INDEXED_EXTRACTIONS = < CSV|W3C|TSV|PSV|JSON >
* Tells Splunk the type of file and the extraction and/or parsing method
Splunk should use on the file.
CSV - Comma separated value format
TSV - Tab-separated value format
PSV - pipe "|" separated value format
W3C - W3C Extended Extended Log File Format
JSON - JavaScript Object Notation format
* These settings default the values of the remaining settings to the
appropriate values for these known formats.
* Defaults to unset.

FIELD_NAMES = [ ,..., ]
* Some CSV and structured files might have missing headers. This attribute
tells Splunk to specify the header field names directly.

TIMESTAMP_FIELDS = [ ,..., ]
* Some CSV and structured files have their timestamp encompass multiple
fields in the event separated by delimiters. This attribue tells Splunk to
specify all such fields which constitute the timestamp in a
comma-separated fashion.
* If not specified, Splunk tries to automatically extract the timestamp of
the event.