JSON file will not break correctly OR create field extractions

[jason_hotchkiss]

inputsHello - 

I have the following log that will not line break using the traditional ([\r\n)+).  Each event splits between:  "Properties": {

Here is what I have tried in my Props.conf:

[ mysourcetype ]
BREAK_ONLY_BEFORE=\"Properties\"\: \{
LINE_BREAKER=^{
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
MAX_EVENTS=40000
SHOULD_LINEMERGE=true
disabled=false
pulldown_type=true

{ "computers": [ { "Properties": { "haslaps": false, "highvalue": false, "name": "DATA", "domain": "DATA", "objectid": "DATA", "distinguishedname": "DATA", "description": null, "enabled": true, "unconstraineddelegation": false, "serviceprincipalnames": [ "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA" ], "lastlogontimestamp": 1501470433, "pwdlastset": 1500622271, "operatingsystem": "DATA" }, "AllowedToDelegate": [], "AllowedToAct": [], "PrimaryGroupSid": "DATA", "Sessions": [], "LocalAdmins": [], "RemoteDesktopUsers": [], "DcomUsers": [], "PSRemoteUsers": [], "ObjectIdentifier": "DATA", "Aces": [ { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": DATA }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": false }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": false }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "DATA", "RightName": "DATA", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Unknown", "RightName": "DATA", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericAll", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Unknown", "RightName": "GenericAll", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "WriteDacl", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "WriteOwner", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericWrite", "AceType": "", "IsInherited": true } ] }, { "Properties": { "haslaps": false, "highvalue": false, "name": "DATA", "domain": "DATA", "objectid": "DATA", "distinguishedname": "DATA", "description": null, "enabled": true, "unconstraineddelegation": false, "serviceprincipalnames": [ "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA", "DATA" ], "lastlogontimestamp": 1506599859, "pwdlastset": 1505682659, "operatingsystem": "DATA" }, "AllowedToDelegate": [], "AllowedToAct": [], "PrimaryGroupSid": "DATA", "Sessions": [], "LocalAdmins": [], "RemoteDesktopUsers": [], "DcomUsers": [], "PSRemoteUsers": [], "ObjectIdentifier": "DATA", "Aces": [ { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "Owner", "AceType": "", "IsInherited": false }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericAll", "AceType": "", "IsInherited": false }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericAll", "AceType": "", "IsInherited": false }, { "PrincipalSID": "DATA", "PrincipalType": "User", "RightName": "GenericAll", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericAll", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Unknown", "RightName": "GenericAll", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "WriteDacl", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "WriteOwner", "AceType": "", "IsInherited": true }, { "PrincipalSID": "DATA", "PrincipalType": "Group", "RightName": "GenericWrite", "AceType": "", "IsInherited": true } ] <...truncated...>

Any suggestions on how I can get this to break properly & extract the field value pairs?  Thank you!

[kamlesh_vaghela · Accepted solution]

@jason_hotchkiss 

Can you please try this?

[YOUR_SOURCETYPE]
SHOULD_LINEMERGE=false
LINE_BREAKER=]}(\,\s){\"Properties
NO_BINARY_CHECK=true
SEDCMD-a=s/{\"computers\": \[//g
SEDCMD-b=s/\]}\]}$/]}/g
TRUNCATE=0

Thanks
Kamlesh Vaghela

[jason_hotchkiss]

This works! Thank you!

[jason_hotchkiss]

@kamlesh_vaghela 

Hello Kamlesh - that did not work, and I believe that was my fault, as the log format came in wrong.

Here is the regex:  https://regex101.com/r/KkpSIM/1

[kamlesh_vaghela]

@jason_hotchkiss 

Ca you please try this?

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=false
LINE_BREAKER=\]\n\s{4}}(,\s{5}){\n\s{6}"Properties":
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false
SEDCMD-a=s/{\n\s{2}"computers":\s\[\n\s{4}//g
SEDCMD-b=s/\n\s*//g
SEDCMD-c=s/\]}\]}$/]}/g

Your provided json was not valid. So I have added closing brackets at the end. 

Check the sample I have used.

https://regex101.com/r/NFxrJp/1

Thanks
KV ▄︻̷̿┻̿═━一

If this reply helps you, an upvote would be appreciated.