Getting Data In

CSV file with last 2 fields XML payloads

odigokid
Engager

Need help with the following CSV (everything I am trying, the XML fields are getting parsed incorrectly)

so I have a CSV file with a header line and then data record

The last two fields - FullRequest, and FullResponse - are SOAP payloads which have \n and ',' in the payload - so splunk is treating the newline as a new event, and it's also chopping at the comma because that's the delimiter.

The other fields before these are what I would call your standard CSV fields in "","","","" - but as you can see some fields can be empty (i.e. ,"",)

so looking for approaches to parsing this log file.

0 Karma

woodcock
Esteemed Legend

I generally use INDEXED_EXTRACTIONS which should work fine for your data:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Extractfieldsfromfileswithstructureddata

0 Karma

odigokid
Engager

Hi - this is my current props.conf which is not working

DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

Image of what I am seeing on search head - the xml is getting broken on the newlines

alt text

0 Karma

odigokid
Engager

Image link - link text

0 Karma

odigokid
Engager
0 Karma

ssadanala1
Contributor

posting a sample event will help

0 Karma

odigokid
Engager

I tried to attach but stated I don't have enough karma points - let me paste here. (I have not put all the data in the payloads due to customer data - but I have put a line there that has , in the data. and you see the "newline's" in the payloads.

LogType(v1.0),RootLogId,SubLogId,TransactionId,Instance,Operation,Status,User,Hostname,Protocol,Target,StartTime,ExecuteTime,ResponseCode,FullRequest,FullResponse
"southbound","PLP1EM01PL61804231005392658CAI3G1_2","/1/1/1","","","PGW_Create","SUCCESSFUL","","PLP1EM01PL6","SOAP","PGW-SNQ","2018-04-23 10.05.39.892","00 00:00:00.843","0","

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

",

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

"

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...