Json line breaking issue

[surekhasplunk]

Hi,

I have exactly same issue as below
https://answers.splunk.com/answers/513703/json-breaking-single-string-into-multiple-events.html

so i added below to my props.conf file :

[_json_source]
CHARSET=UTF-8
DATETIME_CONFIG = CURRENT
KV_MODE = json
TRUNCATE = 0
SEDCMD-fixfooters=s/]}//g
LINE_BREAKER = ([\r\n,]*(?:{[^[{]+[)?){"teamInCharge
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
disabled = false
pulldown_type = true

The events are breaking correctly but the first line is not coming properly, only as raw text and not in json format.

Could you please help.

The different between the raw text between the 1st event and 2nd event i saw is at the end of the line.

1st line ends with below showing other fields along with raw text:
{"teamInCharge":[], bla bla,"serialNumber":""}]

2nd lines ends with below showing syntax highlighted part also
{"teamInCharge":[], bla bla,"serialNumber":""}

Please help.

[to4kawa]

https://answers.splunk.com/answers/513703/json-breaking-single-string-into-multiple-events.html
sample:

| makeresults
| eval _raw="{\"batch_id\":\"0-39-1490386204359\",\"sampling_rate\":1,\"n\":500,\"events\":[{\"earlyAccess\":null,\"profileId\":\"d037d7da-bd83-11e6-80d6-0a2c5cb56663\",\"ip\":\"74.000.000.238\",\"deviceVersion\":[{\"component\":\"app\",\"build\":\"oncue#1.19.37.108208#mallard2_GA.0\",\"otherDetail\":null},{\"component\":\"service\",\"build\":\"qa-a.stb.fios.tv\",\"otherDetail\":null},{\"component\":\"location\",\"build\":\"1000.00,1000.00\",\"otherDetail\":null},{\"component\":\"Client\",\"build\":\"108208#release_signed\",\"otherDetail\":null}],\"__lane\":\"prod\",\"__host\":\"unknown\",\"navigationStack\":[{\"name\":\"Playback\",\"index\":0,\"type\":\"LINEAR\"}],\"platformVersion\":\"108208#release_signed\",\"assetType\":\"LinearAsset\",\"assetId\":\"50281370fd523ecbaafd8f4d8145e006-adf51f1b5b403b6daa88b62d9c8567fa-2017-03-24-1\",\"restrictedBy\":null,\"programStartTime\":1490380200000,\"attributionId\":\"8cca77ff-1480-b037-b0b2-015b01e09338\",\"assetSessionId\":\"1490386121819\",\"__sourceType\":\"device\",\"serviceTimestamp\":null,\"appVersion\":\"1.19.37.108208\",\"playType\":\"tuneIn\",\"collectionId\":\"\",\"deviceType\":\"501\",\"accountId\":\"1004249\",\"playRate\":1000,\"maestro\":{\"vhoId\":\"\",\"host\":\"qa-a-aws.stb.fios.tv\",\"userAgent\":\"Mozilla/5.0 (STB; CPU 501 OS 108208) OnCue/1.19.37\",\"version\":\"4.4.3851\",\"inHome\":true,\"ipAddress\":\"74.000.000.238\"},\"encoderDelay\":124000,\"__eventId\":\"s9a98hR/sMGKvAFbAe90tQ==\",\"__timestamp\":1490386121909,\"sessionId\":\"617F6EA8-1490385232\",\"__eventName\":\"1\",\"programEndTime\":1490390100000,\"programId\":\"adf51f1b5b403b6daa88b62d9c8567fa\",\"__source\":\"unknown\",\"liveTuneType\":\"live\",\"deviceTimestamp\":1490386121819,\"deviceId\":\"617F6EA8\",\"__eventVersion\":16,\"recordingId\":null,\"channelId\":\"50281370fd523ecbaafd8f4d8145e006\",\"timeZone\":\"America/New_York\",\"eventProgramPoint\":1490386121714},{\"earlyAccess\":null,\"profileId\":\"45737761-ac2b-11e6-80d6-0a2c5cb56663\",\"ip\":\"68.000.000.133\",\"deviceVersion\":[{\"component\":\"app\",\"build\":\"oncue#1.19.40.108253#mallard2_GA.0\",\"otherDetail\":null},{\"component\":\"service\",\"build\":\"qa-a.stb.fios.tv\",\"otherDetail\":null},{\"component\":\"location\",\"build\":\"1000.00,1000.00\",\"otherDetail\":null},{\"component\":\"Client\",\"build\":\"108253#release\",\"otherDetail\":null}],\"__lane\":\"prod\",\"__host\":\"unknown\",\"navigationStack\":[{\"name\":\"\",\"index\":0,\"type\":\"\"}],\"platformVersion\":\"108253#release\",\"assetType\":\"LinearAsset\",\"assetId\":\"2732f41bdecc33aca2a23146eabd0954-5e4c3aaa6ef7312b8104c94c842d6a3f-2017-03-24-1\",\"restrictedBy\":null,\"programStartTime\":1490385600000,\"attributionId\":\"ffffffff-ffff-ffff-ffff-fffffffffff\",\"assetSessionId\":\"1490386010685\",\"__sourceType\":\"device\",\"serviceTimestamp\":null,\"appVersion\":\"1.19.40.108253\",\"playType\":\"tuneOut\",\"collectionId\":\"\",\"deviceType\":\"501\",\"accountId\":\"1003469\",\"playRate\":0,\"maestro\":{\"vhoId\":\"\",\"host\":\"qa-a-aws.stb.fios.tv\",\"userAgent\":\"Mozilla/5.0 (STB; CPU 501 OS 108253) OnCue/1.19.40\",\"version\":\"4.4.3851\",\"inHome\":true,\"ipAddress\":\"68.000.000.133\"},\"encoderDelay\":49000,\"__eventId\":\"uXvQcxR/sMGKvAFbAe6R2w==\",\"__timestamp\":1490386063835,\"sessionId\":\"617F7743-1490378565\",\"__eventName\":\"1\",\"programEndTime\":1490387400000,\"programId\":\"5e4c3aaa6ef7312b8104c94c842d6a3f\",\"__source\":\"unknown\",\"liveTuneType\":\"live\",\"deviceTimestamp\":1490386063730,\"deviceId\":\"617F7743\",\"__eventVersion\":16,\"recordingId\":null,\"channelId\":\"2732f41bdecc33aca2a23146eabd0954\",\"timeZone\":\"America/New_York\",\"eventProgramPoint\":1490387400000}]}"
| rex mode=sed "s/.*?\[(.*)]\.*/\1/ s/,({\"earlyAccess)/!\1/"
| makemv delim="!" _raw
| stats count by _raw
| spath

props.conf

SEDCMD-trim = s/.*?\[(.*)]\.*/\1/
SHOULD_LINEMERGE = false
LINE_BREAKER = (,){\"earlyAccess
KV_MODE = JSON

SEDCMD and LINE_BREAKER can be confirmed like this.

You can do it.

[to4kawa]

you already modify raw log. we can't correct this.