topic Re: How to extract csv files with common fields in the header? in Getting Data In

How to extract csv files with common fields in the header?

VatsalJagani — Mon, 08 May 2023 13:14:42 GMT

Below is my CSV file format.

Time Span:,Full Time-span Rate:,Cumulative Scope:,Net This is Table Header Field1,Field2,Field3,Field4 Total1,/,1.20%,2.34%,N/A Total2,/Total2,1.20%,2.05%,N/A Total3,/Total/Total3,1.20%,N/A,N/A Effect4,/Total/Total4,0.00%,N/A,N/A

Here first 3 lines are common fields and values.
4th line is the table header (willing to extract that as a field as well if possible)
The rest is the actual CSV file, I would like to extract it as field value pairs.

Re: How to extract csv files with common fields in the header?

richgalloway — Mon, 08 May 2023 13:55:54 GMT

What have you tried so far? Have you tried something like this in props.conf?

[mysourcetype] INDEXED_EXTRACTIONS = csv HEADER_FIELD_LINE_NUMBER = 4 DATETIME_CONFIG = current

Re: How to extract csv files with common fields in the header?

VatsalJagani — Mon, 08 May 2023 16:42:36 GMT

@richgalloway - This is helpful. Thanks!!!

But I would like to include common fields from the header to all the events, not sure if that is possible.

Re: How to extract csv files with common fields in the header?

richgalloway — Mon, 08 May 2023 16:52:09 GMT

To do that you'll have to write a scripted input that parses the CSV and copies the common fields to each line in the file.

Re: How to extract csv files with common fields in the header?

VatsalJagani — Tue, 09 May 2023 04:27:44 GMT

Yes, Python script is always an option, but I was wondering if it is possible without that. Something like KV_MODE=multi

Re: How to extract csv files with common fields in the header?

richgalloway — Tue, 09 May 2023 12:43:06 GMT

There is no setting that does what you desire. KV_MODE = multi extracts fields from table-formatted data.