Getting Data In

Why is Splunk not showing full JSON data on search?

sdawsonkg
Path Finder

I have a json file that contains 2000+ lines of data, it looks somewhat like this -

[
  {
    "line": 2,
    "elements": [
      {
        "before": [
          {
            "result": {
              "duration": 6692500639,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          },
          {
            "result": {
              "duration": 112646517,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "line": 21,
        "name": "random_name",
        "description": "",
        "id": "random_id",
        "after": [
          {
            "result": {
              "duration": 2054619875,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "type": "scenario",
        "keyword": "Scenario Outline",
        "steps": [
          {
            "result": {
              "duration": 3627081999,
              "status": "passed"
            },
            "line": 5,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "Given "
          },
          {
            "result": {
              "duration": 126013564303,
              "error_message": ""... more than 250 lines of stack trace over here ..."",
              "status": "failed"
            },
            "line": 9,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "some_key"
          }
          }
-------- some more similar lines here ----------
        ],
        "tags": [
          {
            "line": 1,
            "name": "test_name"
          },
          {
            "line": 1,
            "name": "test_name"
          }
        ]
      }
    ],
    "name": "random_name",
    "description": "",
    "id": "random_id",
    "keyword": "Feature",
    "uri": "random_uri",
    "tags": [
      {
        "line": 1,
        "name": "test_name"
      },
      {
        "line": 1,
        "name": "test_name"
      }
    ]
  }
]

Howevere splunk is only showing the first 108 lines from the file and not the whole
The SPL I used is -

index="test" host="localhost" sourcetype="json_data"

The location json file is on the indexer as well as on the forwarder. but nothing works.

Here are my configs for props.conf -

[json_data]
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = ]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
TRUNCATE = 0
NO_BINARY_CHECK = true
CHARSET = UTF-8
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

and what splunk displays -
alt text

Can someone please help.
Thanks!

If you need any other info please comment.

mdsnmss
SplunkTrust
SplunkTrust

Based on your sample JSON I think the issue is with your linebreaking. Your configuration is to linebreak after "]". There are those brackets within the inside of your event already so that isn't something that necessarily dictates being the end of the event. Is there a definitive pattern that shows up near the end of your event every time? Or is this just a single JSON event you are trying to ingest? If it is just a single event you can likely leave MUST_BREAK_AFTER empty as it defaults to empty.

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

You may also want some timestamping settings included as well. TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD, and TIME_FORMAT may be something that is useful to set. Please let me know if you have any questions on these!

0 Karma

sdawsonkg
Path Finder

initially I kept the MUST_BREAK_AFTER as blank. Still it did not show the full data.

Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...