Splunk Search

Why is my JSON regex expression not working properly?

Dawson014
Path Finder

I have a JSON file, which is being indexed by Splunk, the format is like -

{
   testdata : [
      {
          "testid" : 1234,
          "abc" : "def",
          "def" : "abc",
          "httpServer" : [
               {
                     "responseTime" : 300,
                     "responseCode" : 200,
                     "datetime": 0982894965
               },
               {
                    "responseTime": 312,
                    "responseCode": 200,
                    "datetime": 09230948509
                }
           ],
          "transactions" : [
                 {
                   ....
                 },
                 {
                   ....
                 }
            ]
       },
       {
           "testid": 1234,
           ....
       }
   ]
}

Can someone please suggest a regex which can give me relevant data for every "testid". Whatever regex I tried doesn't seem to work. I was using this stanza in my props.conf

[randomsourcetype]
[accountgroups]
TRUNCATE = 0
KV_MODE = json
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
LINE_BREAKER = ([\r\n]*)(?=\{)
DATETIME_CONFIG = CURRENT

Thanks in advance

Tags (3)
1 Solution

felipesewaybric
Contributor

have you try the with the default json sourcetype? Testing in local here, i can access the data:
testdata.testid

View solution in original post

Dawson014
Path Finder

For anyone having the same issue - Just done user LINE_BREAKER or leave it blank or remove it. That's how I got it to work. Thanks for the suggestions.

tkopchak
Path Finder

I have had success using a configuration like this for handling json:

[sourcetype]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = json
NO_BINARY_CHECK = true
category = Structured
disabled = false
TRUNCATE = 999999

Once this data is indexed you can use the mvexpand command to view all values for the testid field.

Noah_Woodcock
Path Finder

My favorite tool for problems like this is Regex101.com
At least take a look 🙂

woodcock
Esteemed Legend

Like this:

|makeresults|eval _raw="{
    testdata : [
       {
           \"testid\" : 1234,
           \"abc\" : \"def\",
           \"def\" : \"abc\",
           \"httpServer\" : [
                {
                      \"responseTime\" : 300,
                      \"responseCode\" : 200,
                      \"datetime\": 0982894965
                },
                {
                     \"responseTime\": 312,
                     \"responseCode\": 200,
                     \"datetime\": 09230948509
                 }
            ],
           \"transactions\" : [
                  {
                    ....
                  },
                  {
                    ....
                  }
             ]
        },
        {
            \"testid\": 1234,
            ....
        }
    ]
 }"
 | rex max_match=0 "\s+{[\r\n]+\s+\"testid\"\s*:\s*(?<testid>\d+)"

felipesewaybric
Contributor

have you try the with the default json sourcetype? Testing in local here, i can access the data:
testdata.testid

Dawson014
Path Finder

turns out if you remove the LINE_BREAKER = it works. Thanks for your suggestion.

0 Karma

MonkeyK
Builder

I have also had success with the json sourcetype.
For more complex json hierarchies, spath works very well.

in addition to the Splunk doc link above, here is an answers refrence that may help
https://answers.splunk.com/answers/63368/how-to-handle-simple-json-array-with-spath.html

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...