Getting Data In

How to do CSV Event Separation?

JGP
Explorer

We want event to separated for each header whenever there is new entry in the csv file. what would be the props applied to the sourcetype to have a single event 

sample file

JGP_0-1690466270798.png

 

want details in one event whenever there is header inserted in csv file

JGP_1-1690466493598.png

please suggest

Labels (1)
Tags (1)
0 Karma

Praz_123
Communicator

@JGP 

Will you please elaborate the query. 

0 Karma

dhruv
Explorer

source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

 

1. source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv" - This is just searching for a particular index, source, sourcetype.

2. | rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0 - 

rex - It extracts fields inline from the events. The field extraction is not permanent. https://docs.splunk.com/Documentation/SplunkCloud/9.0.2305/SearchReference/Rex

"(?<Test_ID>\d+),(?<name>.*)" - It is regular expression as per my sample data. Try https://regex101.com/ for building the regex as per your data.

max_match=0 - It will allow rex command to match all matching field-values. Otherwise rex will only match the first occurrence.

0 Karma

dhruv
Explorer

Hi JGP,

You can create your own custom sourcetype to parse the events separately.
Let me give you an example : 
My Sample Data - 
Test ID,name
1,test1
2,test2
Test ID,name
3,test3
4,test4
Test ID,name
5,test5
6,test6

My Sourcetype configuration - 
[test_csv]
LINE_BREAKER=(\n+)Test
SHOULD_LINEMERGE=true

NOTE - You can define the other parameters as per your need.

This is how it parses the data : 

dhruv_0-1690885965447.png

If you want to extract fields, You can write your own REGEX to do it.
Here is how I did it using "REX" command. Just to showcase : 

dhruv_1-1690886082812.png


The query : 
source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

Hope this helps.

Thanks.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...

Index This | How many sevens are there between 1 and 100?

August 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...