Getting Data In

Why is Splunk not showing full JSON data on search?

sdawsonkg
Path Finder

I have a json file that contains 2000+ lines of data, it looks somewhat like this -

[
  {
    "line": 2,
    "elements": [
      {
        "before": [
          {
            "result": {
              "duration": 6692500639,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          },
          {
            "result": {
              "duration": 112646517,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "line": 21,
        "name": "random_name",
        "description": "",
        "id": "random_id",
        "after": [
          {
            "result": {
              "duration": 2054619875,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "type": "scenario",
        "keyword": "Scenario Outline",
        "steps": [
          {
            "result": {
              "duration": 3627081999,
              "status": "passed"
            },
            "line": 5,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "Given "
          },
          {
            "result": {
              "duration": 126013564303,
              "error_message": ""... more than 250 lines of stack trace over here ..."",
              "status": "failed"
            },
            "line": 9,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "some_key"
          }
          }
-------- some more similar lines here ----------
        ],
        "tags": [
          {
            "line": 1,
            "name": "test_name"
          },
          {
            "line": 1,
            "name": "test_name"
          }
        ]
      }
    ],
    "name": "random_name",
    "description": "",
    "id": "random_id",
    "keyword": "Feature",
    "uri": "random_uri",
    "tags": [
      {
        "line": 1,
        "name": "test_name"
      },
      {
        "line": 1,
        "name": "test_name"
      }
    ]
  }
]

Howevere splunk is only showing the first 108 lines from the file and not the whole
The SPL I used is -

index="test" host="localhost" sourcetype="json_data"

The location json file is on the indexer as well as on the forwarder. but nothing works.

Here are my configs for props.conf -

[json_data]
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = ]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
TRUNCATE = 0
NO_BINARY_CHECK = true
CHARSET = UTF-8
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

and what splunk displays -
alt text

Can someone please help.
Thanks!

If you need any other info please comment.

mdsnmss
SplunkTrust
SplunkTrust

Based on your sample JSON I think the issue is with your linebreaking. Your configuration is to linebreak after "]". There are those brackets within the inside of your event already so that isn't something that necessarily dictates being the end of the event. Is there a definitive pattern that shows up near the end of your event every time? Or is this just a single JSON event you are trying to ingest? If it is just a single event you can likely leave MUST_BREAK_AFTER empty as it defaults to empty.

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

You may also want some timestamping settings included as well. TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD, and TIME_FORMAT may be something that is useful to set. Please let me know if you have any questions on these!

0 Karma

sdawsonkg
Path Finder

initially I kept the MUST_BREAK_AFTER as blank. Still it did not show the full data.

Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...