topic Re: Why is Splunk not showing full JSON data on search? in Getting Data In

Why is Splunk not showing full JSON data on search?

sdawsonkg — Thu, 25 Jan 2018 15:52:12 GMT

I have a json file that contains 2000+ lines of data, it looks somewhat like this -

[
  {
    "line": 2,
    "elements": [
      {
        "before": [
          {
            "result": {
              "duration": 6692500639,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          },
          {
            "result": {
              "duration": 112646517,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "line": 21,
        "name": "random_name",
        "description": "",
        "id": "random_id",
        "after": [
          {
            "result": {
              "duration": 2054619875,
              "status": "passed"
            },
            "match": {
              "location": "some_value"
            }
          }
        ],
        "type": "scenario",
        "keyword": "Scenario Outline",
        "steps": [
          {
            "result": {
              "duration": 3627081999,
              "status": "passed"
            },
            "line": 5,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "Given "
          },
          {
            "result": {
              "duration": 126013564303,
              "error_message": ""... more than 250 lines of stack trace over here ..."",
              "status": "failed"
            },
            "line": 9,
            "name": "random_name",
            "match": {
              "location": "some_value"
            },
            "keyword": "some_key"
          }
          }
-------- some more similar lines here ----------
        ],
        "tags": [
          {
            "line": 1,
            "name": "test_name"
          },
          {
            "line": 1,
            "name": "test_name"
          }
        ]
      }
    ],
    "name": "random_name",
    "description": "",
    "id": "random_id",
    "keyword": "Feature",
    "uri": "random_uri",
    "tags": [
      {
        "line": 1,
        "name": "test_name"
      },
      {
        "line": 1,
        "name": "test_name"
      }
    ]
  }
]

Howevere splunk is only showing the first 108 lines from the file and not the whole
The SPL I used is -

index="test" host="localhost" sourcetype="json_data"

The location json file is on the indexer as well as on the forwarder. but nothing works.

Here are my configs for props.conf -

[json_data]
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = ]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
TRUNCATE = 0
NO_BINARY_CHECK = true
CHARSET = UTF-8
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

and what splunk displays -

Can someone please help.
Thanks!

If you need any other info please comment.

Re: Why is Splunk not showing full JSON data on search?

mdsnmss — Tue, 29 Sep 2020 17:50:52 GMT

Based on your sample JSON I think the issue is with your linebreaking. Your configuration is to linebreak after "]". There are those brackets within the inside of your event already so that isn't something that necessarily dictates being the end of the event. Is there a definitive pattern that shows up near the end of your event every time? Or is this just a single JSON event you are trying to ingest? If it is just a single event you can likely leave MUST_BREAK_AFTER empty as it defaults to empty.

Re: Why is Splunk not showing full JSON data on search?

mdsnmss — Tue, 29 Sep 2020 17:50:55 GMT

You may also want some timestamping settings included as well. TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD, and TIME_FORMAT may be something that is useful to set. Please let me know if you have any questions on these!

Re: Why is Splunk not showing full JSON data on search?

sdawsonkg — Tue, 29 Sep 2020 17:51:20 GMT

initially I kept the MUST_BREAK_AFTER as blank. Still it did not show the full data.