Splunk Search

Why is CSV data with multiple values in one cell parsing incorrectly?

kimle
Engager

I'm trying to upload a CSV file into Splunk, however, it doesn't seem to parse it correctly for the multiple values fields.
Here is the data I have:

rule,source_address, dest_address, proto
Test, 1.1.1.1 1.2.2.2 1.3.3.3, 4.4.4.4 4.4.5.5, icmp https

I want all the source address/dest address/application in a separate row so that they are independent for searching, but I couldn't get it to work. After I uploaded the file, if I looked at the extracted field for source_address, it will show all three in 1 line (e.g source_address="1.1.1.1 1.2.2.2 1.3.3.3") instead of 3 unique values

Desire output when running " index=test | table rule, source_address, dest_address, proto" without EVAL, makemv, etc:

alt text

For now, I have:
alt text

I have tried modifying props.conf and transforms.conf such as the following and other suggestions I found in https://answers.splunk.com/, but nothing seems to work.

props.conf:
[my_sourcetype]
DELIMS = ","
FIELDS = rule, source_address, dest_address, proto
REPORT-extract_space = extract_space

transforms.conf
[extract_space]
DELIMS = "  "
FIELDS = source_address, dest_address, proto

How can I get these columns to parse correctly? Please help!

Thanks in advance!

0 Karma
1 Solution

kimle
Engager

I found a solution to my issue. I found an article abut fields.conf from https://docs.splunk.com/Documentation/Splunk/8.0.0/Knowledge/ConfigureSplunktoparsemulti-valuefields

Using what suggested in the docs, I created fields.conf file in $SPLUNK_HOME/etc/system/local/. In this file, I added

[my_fieldname]
TOKENIZER = \S+

You can create a regex pattern to look to IPs, but in my case all of my IPs are separated by space, so I used the regex to get anything that is not non-whitespace and did the trick!

Thanks everyone for helping! : )

View solution in original post

0 Karma

kimle
Engager

I found a solution to my issue. I found an article abut fields.conf from https://docs.splunk.com/Documentation/Splunk/8.0.0/Knowledge/ConfigureSplunktoparsemulti-valuefields

Using what suggested in the docs, I created fields.conf file in $SPLUNK_HOME/etc/system/local/. In this file, I added

[my_fieldname]
TOKENIZER = \S+

You can create a regex pattern to look to IPs, but in my case all of my IPs are separated by space, so I used the regex to get anything that is not non-whitespace and did the trick!

Thanks everyone for helping! : )

0 Karma

nikita_p
Contributor

Hi kimle,

While uploading .csv file as a lookup in Splunk, it should be in a proper csv format. You should have your field in source_address and dest_address one after the other and not in a single sentence. The CSV file cannot contain non-utf-8 characters. Plain ascii text is supported, as is any character set that is also valid utf-8. And then you can try uploading your csv as a lookup and check the result.
Let me know if you have any questions.

0 Karma

okayal
Explorer

I suspect it's the type of CSV you are importing. If you have tried saving the file in MS excel, you will see multiple options, csv (comma delimited), csv (Macintosh) and csv (MS-DOS). They differ in the delimiter type and line terminator, and IIRC the comma delimited one is the only one that works with Splunk out of the box.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...