Getting Data In

Importing CSV file without a header

AnujaJ
Path Finder

I have a "!" seperated file without a header. I want to import it in Splunk. However Splunk by default takes the first event as the header and all other events below. I want to manually name the fields in the sourcetype. I was wondering if this is possible.

So far, my sourcetype looks like this:

[ ca_csv ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
DELIMS=!
FIELDS=a1,a2,a3,a4,a5
FIELD_DELIMITER=!
category=Custom
disabled=false
pulldown_type=true
CHECK_FOR_HEADER=False

However, this does not rename the fields as a1,a2,a3,a4,a5. I have 5 fields per event.

Sample data

L01!0112!11493!20191111000012!1149385630101120002012812019111032019111020191110690952404800415;201911
L02!0112!11493!20191111000012!0003M00BF000001010020191111000012D823AIB000000bR0FFF0001
L03!0112!114938563!20191111000013!0003M0036010001000020191110230005D823O07F L04!0112!114938563!20191111000014!025092664050002011201281201911111000114

Gregski11
Contributor

i have the exact same issue, doing a one time import of a simple four row text test CSV file without a header and Splunk insists the top row is the header which it is not, I am a web interface GUI junkie so would love to know how to fix this using the GUI on a Windows platform not Unix not editing any wild text files

0 Karma

gfreitas
Builder

I believe you can use transforms.conf for that. See this link: https://answers.splunk.com/answers/170251/how-to-extract-two-fields-separated-by-delimiter-c.html

props.conf would look like:

[ca_csv]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
category=Custom
disabled=false
pulldown_type=true
REPORT-fields=csv_fields

Then on transforms.conf

[csv_fields]
DELIMS = !
FIELDS = a1,a2,a3,a4,a5

AnujaJ
Path Finder

I am uploading the file manually for testing but these settings do not work. Also without FIELD_DELIMITER there is no recognition of different fields. Is it possible to see the effect on manually uploaded file?

0 Karma

gfreitas
Builder

This REPORT-fields will work on search time. Try to create a temporary index and import the file with those settings and see if it works. I tried on my lab and seems to work fine.

0 Karma

niketn
Legend

@AnujaJ will it be possible to add some sample data with couple of rows for the community to assist you better. Please mock/anonymize any sensitive information. Splunk does allow you to provide your own set of comma separated field names instead of using header row.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...