Can you help me with a line breaking issue in a Multi-line JSON file input with an array of records?

reed_kelly — Tue, 29 Sep 2020 22:40:45 GMT

I have input files from MS Graph with pretty-printed JSON that looks something like the following (ellipses used liberally...). I am unable to find the right LINE_BREAKER value or BREAK_ONLY_BEFORE or BREAK_ONLY_AFTER to split the records on the comma between the }, and the {. Note that this sample has had the indentation with extra spaces removed.

    [
        {
            "@odata.type":  "#Microsoft.graph....",
            "id":  "...",
            "...":  "...",
            "foobar":  { 
                                  "foo1":  "bar1",
                                  "foo2":  "bar2",
                            },
            "...":  "...",
            "barfoo":  {
                                  "bar1":  "foo1",
                                  "bar2":  "foo2",
                            }
        },
        {
            "@odata.type":  "#Microsoft.graph....",
            "id":  "...",
            "...":  "...",
        },
        {
            "...":  "...",
            "...":  "...",
        }
    ]

This props.conf fails, because there are other }, strings within each record (see end of "foobar"):

[json]
TRUNCATE = 0
KV_MODE = json
TIME_PREFIX = \"xxxEventDateTime\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
MUST_BREAK_AFTER = \s*}\,

This is for a 6.5.x Splunk hwf feeding a 6.5.x indexer cluster.

Re: Can you help me with a line breaking issue in a Multi-line JSON file input with an array of records?

reed_kelly — Wed, 09 Jan 2019 18:44:43 GMT

I found the following to work:

TRUNCATE = 0
SHOULD_LINEMERGE = false
PREAMBLE_REGEX = ^\s*\[\s*$
LINE_BREAKER = }(,\s*[\r\n]*\s*){

topic Re: Can you help me with a line breaking issue in a Multi-line JSON file input with an array of records? in Getting Data In

Can you help me with a line breaking issue in a Multi-line JSON file input with an array of records?

Re: Can you help me with a line breaking issue in a Multi-line JSON file input with an array of records?