Splunk Search

Why is CSV data with multiple values in one cell parsing incorrectly?

kimle
Engager

I'm trying to upload a CSV file into Splunk, however, it doesn't seem to parse it correctly for the multiple values fields.
Here is the data I have:

rule,source_address, dest_address, proto
Test, 1.1.1.1 1.2.2.2 1.3.3.3, 4.4.4.4 4.4.5.5, icmp https

I want all the source address/dest address/application in a separate row so that they are independent for searching, but I couldn't get it to work. After I uploaded the file, if I looked at the extracted field for source_address, it will show all three in 1 line (e.g source_address="1.1.1.1 1.2.2.2 1.3.3.3") instead of 3 unique values

Desire output when running " index=test | table rule, source_address, dest_address, proto" without EVAL, makemv, etc:

alt text

For now, I have:
alt text

I have tried modifying props.conf and transforms.conf such as the following and other suggestions I found in https://answers.splunk.com/, but nothing seems to work.

props.conf:
[my_sourcetype]
DELIMS = ","
FIELDS = rule, source_address, dest_address, proto
REPORT-extract_space = extract_space

transforms.conf
[extract_space]
DELIMS = "  "
FIELDS = source_address, dest_address, proto

How can I get these columns to parse correctly? Please help!

Thanks in advance!

0 Karma
1 Solution

kimle
Engager

I found a solution to my issue. I found an article abut fields.conf from https://docs.splunk.com/Documentation/Splunk/8.0.0/Knowledge/ConfigureSplunktoparsemulti-valuefields

Using what suggested in the docs, I created fields.conf file in $SPLUNK_HOME/etc/system/local/. In this file, I added

[my_fieldname]
TOKENIZER = \S+

You can create a regex pattern to look to IPs, but in my case all of my IPs are separated by space, so I used the regex to get anything that is not non-whitespace and did the trick!

Thanks everyone for helping! : )

View solution in original post

0 Karma

kimle
Engager

I found a solution to my issue. I found an article abut fields.conf from https://docs.splunk.com/Documentation/Splunk/8.0.0/Knowledge/ConfigureSplunktoparsemulti-valuefields

Using what suggested in the docs, I created fields.conf file in $SPLUNK_HOME/etc/system/local/. In this file, I added

[my_fieldname]
TOKENIZER = \S+

You can create a regex pattern to look to IPs, but in my case all of my IPs are separated by space, so I used the regex to get anything that is not non-whitespace and did the trick!

Thanks everyone for helping! : )

0 Karma

nikita_p
Contributor

Hi kimle,

While uploading .csv file as a lookup in Splunk, it should be in a proper csv format. You should have your field in source_address and dest_address one after the other and not in a single sentence. The CSV file cannot contain non-utf-8 characters. Plain ascii text is supported, as is any character set that is also valid utf-8. And then you can try uploading your csv as a lookup and check the result.
Let me know if you have any questions.

0 Karma

okayal
Explorer

I suspect it's the type of CSV you are importing. If you have tried saving the file in MS excel, you will see multiple options, csv (comma delimited), csv (Macintosh) and csv (MS-DOS). They differ in the delimiter type and line terminator, and IIRC the comma delimited one is the only one that works with Splunk out of the box.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...