Getting Data In

CSV file with last 2 fields XML payloads

odigokid
Engager

Need help with the following CSV (everything I am trying, the XML fields are getting parsed incorrectly)

so I have a CSV file with a header line and then data record

The last two fields - FullRequest, and FullResponse - are SOAP payloads which have \n and ',' in the payload - so splunk is treating the newline as a new event, and it's also chopping at the comma because that's the delimiter.

The other fields before these are what I would call your standard CSV fields in "","","","" - but as you can see some fields can be empty (i.e. ,"",)

so looking for approaches to parsing this log file.

0 Karma

woodcock
Esteemed Legend

I generally use INDEXED_EXTRACTIONS which should work fine for your data:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Extractfieldsfromfileswithstructureddata

0 Karma

odigokid
Engager

Hi - this is my current props.conf which is not working

DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

Image of what I am seeing on search head - the xml is getting broken on the newlines

alt text

0 Karma

odigokid
Engager

Image link - link text

0 Karma

odigokid
Engager
0 Karma

ssadanala1
Contributor

posting a sample event will help

0 Karma

odigokid
Engager

I tried to attach but stated I don't have enough karma points - let me paste here. (I have not put all the data in the payloads due to customer data - but I have put a line there that has , in the data. and you see the "newline's" in the payloads.

LogType(v1.0),RootLogId,SubLogId,TransactionId,Instance,Operation,Status,User,Hostname,Protocol,Target,StartTime,ExecuteTime,ResponseCode,FullRequest,FullResponse
"southbound","PLP1EM01PL61804231005392658CAI3G1_2","/1/1/1","","","PGW_Create","SUCCESSFUL","","PLP1EM01PL6","SOAP","PGW-SNQ","2018-04-23 10.05.39.892","00 00:00:00.843","0","

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

",

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

"

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...