Getting Data In

Line breaks within a CSV field.

ocallender
Explorer

I have a .csv file with several fields. there are many date fields and text fields, but fields are long blobs of text (such as the body of an e-mail) lets call such a field "longtext". The problem is that whenever Splunk encounters a newline character in the longtext field, it interprets it as a line break which throws off the event breaks.

How can I get all of the text in "longtext" to be properly indexed without Splunk interpreting the newline as a line break?

Example:

create_time,request_id,username,longtext,responded_time,closed_time
2013-11-23 11:00,2322,johnsmith,Here is the long blob of text I was talking about. If i have a newline here: <newline> 
Splunk sees it as a break in the log file and doesn't place the rest of this text in the longtext field,2013-11-23 13:43,2013-11-23 14:05

Any ideas?

0 Karma

mrsprague
New Member

If this is a DOS format text file, you should be able break on CR-LF line breaks with LINE_BREAKER (in props.conf)

[yoursourcetype]
LINE_BREAKER=(\r\n)
0 Karma

somesoni2
Revered Legend

Following seems to be working for me, for the sample data you have given (to be added in props.conf)

[yoursourcetype]
INDEXED_EXTRACTIONS = csv
KV_MODE = none
MAX_TIMESTAMP_LOOKAHEAD = 50
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
0 Karma
Get Updates on the Splunk Community!

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...

September Community Champions: A Shoutout to Our Contributors!

As we close the books on another fantastic month, we want to take a moment to celebrate the people who are the ...

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...