Getting Data In

How to handle parsing of CSV files with fields containing commas?

bb2324oo
Engager

My data is parsed into CSV files with default comma delimiters.

Many fields contain one or more commas.

When I use this in props.conf:

    [your_sourcetype]
   EXTRACT-second_round = ^(?[^,]*),(?[^,]*),(?[^,]*),(?.*),(?[^,]*),(?[^,]*),(?[^,]*)

However, Commas could only be in field 4, it does not support multiple commas in fields.

Some may propose this approach:
props.conf

[your_sourcetype]
REPORT-blah = hash_delim

transforms.conf

[hash_delim]
DELIMS = "#"
FIELDS = field1, field2, field3 etc

However, I cannot find a delim which must not be in any fields.

Please help. Any suggestion?

Also, why would commas inside the double quotation marks will be considered as a delimiter in a csv?

fdi01
Motivator

try this :

[hash_delim]
DELIMS = ","
FIELDS = field1, field2, field3 etc

see this example below, Extract comma-delimited values into fields:

[extract_csv]
DELIMS = ","
FIELDS = "field1", "field2", "field3"

This example assigns the extracted values from _raw to field1, field2 and field3 (in order of extraction). If more than three values are extracted, the values without a matching field name are ignored.

Get Updates on the Splunk Community!

Update Your SOAR Apps for Python 3.13: What Community Developers Need to Know

To Community SOAR App Developers - we're reaching out with an important update regarding Python 3.9's ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...

Automatic Discovery Part 2: Setup and Best Practices

In Part 1 of this series, we covered what Automatic Discovery is and why it’s critical for observability at ...