Getting Data In

CSV file with last 2 fields XML payloads

Engager

Need help with the following CSV (everything I am trying, the XML fields are getting parsed incorrectly)

so I have a CSV file with a header line and then data record

The last two fields - FullRequest, and FullResponse - are SOAP payloads which have \n and ',' in the payload - so splunk is treating the newline as a new event, and it's also chopping at the comma because that's the delimiter.

The other fields before these are what I would call your standard CSV fields in "","","","" - but as you can see some fields can be empty (i.e. ,"",)

so looking for approaches to parsing this log file.

0 Karma

Esteemed Legend

I generally use INDEXED_EXTRACTIONS which should work fine for your data:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Extractfieldsfromfileswithstructureddata

0 Karma

Engager

Hi - this is my current props.conf which is not working

DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

Image of what I am seeing on search head - the xml is getting broken on the newlines

alt text

0 Karma

Engager

Image link - link text

0 Karma

Engager
0 Karma

Contributor

posting a sample event will help

0 Karma

Engager

I tried to attach but stated I don't have enough karma points - let me paste here. (I have not put all the data in the payloads due to customer data - but I have put a line there that has , in the data. and you see the "newline's" in the payloads.

LogType(v1.0),RootLogId,SubLogId,TransactionId,Instance,Operation,Status,User,Hostname,Protocol,Target,StartTime,ExecuteTime,ResponseCode,FullRequest,FullResponse
"southbound","PLP1EM01PL61804231005392658CAI3G1_2","/1/1/1","","","PGW_Create","SUCCESSFUL","","PLP1EM01PL6","SOAP","PGW-SNQ","2018-04-23 10.05.39.892","00 00:00:00.843","0","

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

",

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

"

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!