Getting Data In

How to do CSV Event Separation?

JGP
Explorer

We want event to separated for each header whenever there is new entry in the csv file. what would be the props applied to the sourcetype to have a single event 

sample file

JGP_0-1690466270798.png

 

want details in one event whenever there is header inserted in csv file

JGP_1-1690466493598.png

please suggest

Labels (1)
Tags (1)
0 Karma

Praz_123
Communicator

@JGP 

Will you please elaborate the query. 

0 Karma

dhruv
Explorer

source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

 

1. source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv" - This is just searching for a particular index, source, sourcetype.

2. | rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0 - 

rex - It extracts fields inline from the events. The field extraction is not permanent. https://docs.splunk.com/Documentation/SplunkCloud/9.0.2305/SearchReference/Rex

"(?<Test_ID>\d+),(?<name>.*)" - It is regular expression as per my sample data. Try https://regex101.com/ for building the regex as per your data.

max_match=0 - It will allow rex command to match all matching field-values. Otherwise rex will only match the first occurrence.

0 Karma

dhruv
Explorer

Hi JGP,

You can create your own custom sourcetype to parse the events separately.
Let me give you an example : 
My Sample Data - 
Test ID,name
1,test1
2,test2
Test ID,name
3,test3
4,test4
Test ID,name
5,test5
6,test6

My Sourcetype configuration - 
[test_csv]
LINE_BREAKER=(\n+)Test
SHOULD_LINEMERGE=true

NOTE - You can define the other parameters as per your need.

This is how it parses the data : 

dhruv_0-1690885965447.png

If you want to extract fields, You can write your own REGEX to do it.
Here is how I did it using "REX" command. Just to showcase : 

dhruv_1-1690886082812.png


The query : 
source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

Hope this helps.

Thanks.

0 Karma
Get Updates on the Splunk Community!

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...

[Live Demo] Watch SOC transformation in action with the reimagined Splunk Enterprise ...

Overwhelmed SOC? Splunk ES Has Your Back Tool sprawl, alert fatigue, and endless context switching are making ...

What’s New & Next in Splunk SOAR

Security teams today are dealing with more alerts, more tools, and more pressure than ever.  Join us on ...