Getting Data In

Importing CSV file without a header

AnujaJ
Path Finder

I have a "!" seperated file without a header. I want to import it in Splunk. However Splunk by default takes the first event as the header and all other events below. I want to manually name the fields in the sourcetype. I was wondering if this is possible.

So far, my sourcetype looks like this:

[ ca_csv ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
DELIMS=!
FIELDS=a1,a2,a3,a4,a5
FIELD_DELIMITER=!
category=Custom
disabled=false
pulldown_type=true
CHECK_FOR_HEADER=False

However, this does not rename the fields as a1,a2,a3,a4,a5. I have 5 fields per event.

Sample data

L01!0112!11493!20191111000012!1149385630101120002012812019111032019111020191110690952404800415;201911
L02!0112!11493!20191111000012!0003M00BF000001010020191111000012D823AIB000000bR0FFF0001
L03!0112!114938563!20191111000013!0003M0036010001000020191110230005D823O07F L04!0112!114938563!20191111000014!025092664050002011201281201911111000114

0 Karma

gfreitas
Builder

I believe you can use transforms.conf for that. See this link: https://answers.splunk.com/answers/170251/how-to-extract-two-fields-separated-by-delimiter-c.html

props.conf would look like:

[ca_csv]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
category=Custom
disabled=false
pulldown_type=true
REPORT-fields=csv_fields

Then on transforms.conf

[csv_fields]
DELIMS = !
FIELDS = a1,a2,a3,a4,a5

AnujaJ
Path Finder

I am uploading the file manually for testing but these settings do not work. Also without FIELD_DELIMITER there is no recognition of different fields. Is it possible to see the effect on manually uploaded file?

0 Karma

gfreitas
Builder

This REPORT-fields will work on search time. Try to create a temporary index and import the file with those settings and see if it works. I tried on my lab and seems to work fine.

0 Karma

niketn
Legend

@AnujaJ will it be possible to add some sample data with couple of rows for the community to assist you better. Please mock/anonymize any sensitive information. Splunk does allow you to provide your own set of comma separated field names instead of using header row.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...