Getting Data In

Importing CSV file without a header

AnujaJ
Path Finder

I have a "!" seperated file without a header. I want to import it in Splunk. However Splunk by default takes the first event as the header and all other events below. I want to manually name the fields in the sourcetype. I was wondering if this is possible.

So far, my sourcetype looks like this:

[ ca_csv ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
DELIMS=!
FIELDS=a1,a2,a3,a4,a5
FIELD_DELIMITER=!
category=Custom
disabled=false
pulldown_type=true
CHECK_FOR_HEADER=False

However, this does not rename the fields as a1,a2,a3,a4,a5. I have 5 fields per event.

Sample data

L01!0112!11493!20191111000012!1149385630101120002012812019111032019111020191110690952404800415;201911
L02!0112!11493!20191111000012!0003M00BF000001010020191111000012D823AIB000000bR0FFF0001
L03!0112!114938563!20191111000013!0003M0036010001000020191110230005D823O07F L04!0112!114938563!20191111000014!025092664050002011201281201911111000114

0 Karma

gfreitas
Builder

I believe you can use transforms.conf for that. See this link: https://answers.splunk.com/answers/170251/how-to-extract-two-fields-separated-by-delimiter-c.html

props.conf would look like:

[ca_csv]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
category=Custom
disabled=false
pulldown_type=true
REPORT-fields=csv_fields

Then on transforms.conf

[csv_fields]
DELIMS = !
FIELDS = a1,a2,a3,a4,a5

AnujaJ
Path Finder

I am uploading the file manually for testing but these settings do not work. Also without FIELD_DELIMITER there is no recognition of different fields. Is it possible to see the effect on manually uploaded file?

0 Karma

gfreitas
Builder

This REPORT-fields will work on search time. Try to create a temporary index and import the file with those settings and see if it works. I tried on my lab and seems to work fine.

0 Karma

niketn
Legend

@AnujaJ will it be possible to add some sample data with couple of rows for the community to assist you better. Please mock/anonymize any sensitive information. Splunk does allow you to provide your own set of comma separated field names instead of using header row.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...