Getting Data In

How to do CSV Event Separation?

JGP
Explorer

We want event to separated for each header whenever there is new entry in the csv file. what would be the props applied to the sourcetype to have a single event 

sample file

JGP_0-1690466270798.png

 

want details in one event whenever there is header inserted in csv file

JGP_1-1690466493598.png

please suggest

Labels (1)
Tags (1)
0 Karma

Praz_123
Communicator

@JGP 

Will you please elaborate the query. 

0 Karma

dhruv
Explorer

source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

 

1. source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv" - This is just searching for a particular index, source, sourcetype.

2. | rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0 - 

rex - It extracts fields inline from the events. The field extraction is not permanent. https://docs.splunk.com/Documentation/SplunkCloud/9.0.2305/SearchReference/Rex

"(?<Test_ID>\d+),(?<name>.*)" - It is regular expression as per my sample data. Try https://regex101.com/ for building the regex as per your data.

max_match=0 - It will allow rex command to match all matching field-values. Otherwise rex will only match the first occurrence.

0 Karma

dhruv
Explorer

Hi JGP,

You can create your own custom sourcetype to parse the events separately.
Let me give you an example : 
My Sample Data - 
Test ID,name
1,test1
2,test2
Test ID,name
3,test3
4,test4
Test ID,name
5,test5
6,test6

My Sourcetype configuration - 
[test_csv]
LINE_BREAKER=(\n+)Test
SHOULD_LINEMERGE=true

NOTE - You can define the other parameters as per your need.

This is how it parses the data : 

dhruv_0-1690885965447.png

If you want to extract fields, You can write your own REGEX to do it.
Here is how I did it using "REX" command. Just to showcase : 

dhruv_1-1690886082812.png


The query : 
source="testing.csv" host="so1" index="test_csv" sourcetype="test_csv"
| rex "(?<Test_ID>\d+),(?<name>.*)" max_match=0
| table Test_ID,name

Hope this helps.

Thanks.

0 Karma
Get Updates on the Splunk Community!

Splunk MCP & Agentic AI: Machine Data Without Limits

  Discover how the Splunk Model Context Protocol (MCP) Server can revolutionize the way your organization ...

Finding Based Detections General Availability

Overview  We’ve come a long way, folks, but here in Enterprise Security 8.4 I’m happy to announce Finding ...

Get Your Hands Dirty (and Your Shoes Comfy): The Splunk Experience

Hands-On Learning and Technical Seminars  Sometimes, you just need to see the code. For those looking for a ...