Why is my JSON regex expression not working properly?

[Dawson014]

I have a JSON file, which is being indexed by Splunk, the format is like -

{
   testdata : [
      {
          "testid" : 1234,
          "abc" : "def",
          "def" : "abc",
          "httpServer" : [
               {
                     "responseTime" : 300,
                     "responseCode" : 200,
                     "datetime": 0982894965
               },
               {
                    "responseTime": 312,
                    "responseCode": 200,
                    "datetime": 09230948509
                }
           ],
          "transactions" : [
                 {
                   ....
                 },
                 {
                   ....
                 }
            ]
       },
       {
           "testid": 1234,
           ....
       }
   ]
}

Can someone please suggest a regex which can give me relevant data for every "testid". Whatever regex I tried doesn't seem to work. I was using this stanza in my props.conf

[randomsourcetype]
[accountgroups]
TRUNCATE = 0
KV_MODE = json
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
LINE_BREAKER = ([\r\n]*)(?=\{)
DATETIME_CONFIG = CURRENT

Thanks in advance

[Dawson014]

For anyone having the same issue - Just done user LINE_BREAKER or leave it blank or remove it. That's how I got it to work. Thanks for the suggestions.

[tkopchak]

I have had success using a configuration like this for handling json:

[sourcetype]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = json
NO_BINARY_CHECK = true
category = Structured
disabled = false
TRUNCATE = 999999

Once this data is indexed you can use the mvexpand command to view all values for the testid field.

[Noah_Woodcock]

My favorite tool for problems like this is Regex101.com
At least take a look 🙂

[woodcock]

Like this:

|makeresults|eval _raw="{
    testdata : [
       {
           \"testid\" : 1234,
           \"abc\" : \"def\",
           \"def\" : \"abc\",
           \"httpServer\" : [
                {
                      \"responseTime\" : 300,
                      \"responseCode\" : 200,
                      \"datetime\": 0982894965
                },
                {
                     \"responseTime\": 312,
                     \"responseCode\": 200,
                     \"datetime\": 09230948509
                 }
            ],
           \"transactions\" : [
                  {
                    ....
                  },
                  {
                    ....
                  }
             ]
        },
        {
            \"testid\": 1234,
            ....
        }
    ]
 }"
 | rex max_match=0 "\s+{[\r\n]+\s+\"testid\"\s*:\s*(?<testid>\d+)"

[felipesewaybric · Accepted solution]

have you try the with the default json sourcetype? Testing in local here, i can access the data:
testdata.testid

[Dawson014]

turns out if you remove the LINE_BREAKER = it works. Thanks for your suggestion.

[MonkeyK]

I have also had success with the json sourcetype.
For more complex json hierarchies, spath works very well.

in addition to the Splunk doc link above, here is an answers refrence that may help
https://answers.splunk.com/answers/63368/how-to-handle-simple-json-array-with-spath.html