Getting Data In
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question

Splunk cannot correctly parse and ingest json event data

hunters_splunk
Splunk Employee
Splunk Employee
‎05-30-2016 10:56 AM

Splunk cannot correctly parse and ingest the following json event data. I have tried all the line break settings but no luck. Thanks in advance for the help. 

{
  "batchcomplete": "",
  "continue": {
    "lecontinue": "20160516100036|18",
    "continue": "-||"
  },
  "query": {
    "logevents": [
      {
        "logid": 28,
        "ns": 0,
        "title": "Wikitext",
        "pageid": 7,
        "logpage": 7,
        "params": {
          "curid": 29,
          "previd": 28,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
         "timestamp": "2016-05-17T13:48:51Z",
        "comment": ""
       },
      {
        "logid": 27,
        "ns": 0,
        "title": "Wikitext",
        "pageid": 7,
        "logpage": 7,
        "params": {
          "curid": 28,
          "previd": 25,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-17T13:47:30Z",
        "comment": ""
      },
      {
        "logid": 26,
        "ns": 10,
        "title": "Template:Thank You",
        "pageid": 12,
        "logpage": 12,
        "params": {
          "curid": 27,
          "previd": 0,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-17T13:45:04Z",
        "comment": ""
      },
      {
        "logid": 25,
        "ns": 0,
        "title": "Wikitext/Table",
        "pageid": 11,
        "logpage": 7,
        "params": {
          "target_ns": 0,
          "target_title": "Wikitext"
        },
        "type": "move",
        "action": "move_redir",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:38:06Z",
        "comment": ""
      },
      {
        "logid": 24,
        "ns": 0,
        "title": "Wikitext",
        "pageid": 7,
        "logpage": 7,
        "params": {
          "target_ns": 0,
          "target_title": "Wikitext/Table"
        },
        "type": "move",
        "action": "move",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:37:49Z",
        "comment": ""
      },
      {
        "logid": 23,
        "ns": 0,
        "title": "Wikitext/Tables",
        "pageid": 9,
        "logpage": 9,
        "params": {
          "curid": 22,
          "previd": 0,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:31:36Z",
        "comment": ""
      },
      {
        "logid": 22,
        "ns": 0,
        "title": "Wikitext",
        "pageid": 7,
        "logpage": 7,
        "params": {
          "curid": 21,
          "previd": 19,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:18:38Z",
        "comment": ""
      },
      {
        "logid": 21,
        "ns": 10,
        "title": "Template:Test",
        "pageid": 0,
        "logpage": 8,
        "params": {},
        "type": "delete",
        "action": "delete",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:16:40Z",
        "comment": "content was: \"test\" (and the only contributor was \"[[Special:Contributions/JohnDoe|JohnDoe]]\")"
      },
      {
        "logid": 20,
        "ns": 10,
        "title": "Template:Test",
        "pageid": 0,
        "logpage": 8,
        "params": {
          "curid": 20,
          "previd": 0,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:15:13Z",
        "comment": ""
      },
      {
        "logid": 19,
        "ns": 0,
        "title": "Wikitext",
        "pageid": 7,
        "logpage": 7,
        "params": {
          "curid": 19,
          "previd": 18,
          "auto": ""
        },
        "type": "patrol",
        "action": "patrol",
        "user": "JohnDoe",
        "timestamp": "2016-05-16T10:14:55Z",
        "comment": ""
      }
    ]
  }
}
Tags (1)
0 Karma
Reply

woodcock
Esteemed Legend
‎05-31-2016 05:43 AM

Just pass it as-is to spath, like this:

|noop|stats count as _raw| eval _raw="{
   \"batchcomplete\": \"\",
   \"continue\": {
     \"lecontinue\": \"20160516100036|18\",
     \"continue\": \"-||\"
   },
   \"query\": {
     \"logevents\": [
       {
         \"logid\": 28,
         \"ns\": 0,
         \"title\": \"Wikitext\",
         \"pageid\": 7,
         \"logpage\": 7,
         \"params\": {
           \"curid\": 29,
           \"previd\": 28,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
          \"timestamp\": \"2016-05-17T13:48:51Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 27,
         \"ns\": 0,
         \"title\": \"Wikitext\",
         \"pageid\": 7,
         \"logpage\": 7,
         \"params\": {
           \"curid\": 28,
           \"previd\": 25,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-17T13:47:30Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 26,
         \"ns\": 10,
         \"title\": \"Template:Thank You\",
         \"pageid\": 12,
         \"logpage\": 12,
         \"params\": {
           \"curid\": 27,
           \"previd\": 0,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-17T13:45:04Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 25,
         \"ns\": 0,
         \"title\": \"Wikitext/Table\",
         \"pageid\": 11,
         \"logpage\": 7,
         \"params\": {
           \"target_ns\": 0,
           \"target_title\": \"Wikitext\"
         },
         \"type\": \"move\",
         \"action\": \"move_redir\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:38:06Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 24,
         \"ns\": 0,
         \"title\": \"Wikitext\",
         \"pageid\": 7,
         \"logpage\": 7,
         \"params\": {
           \"target_ns\": 0,
           \"target_title\": \"Wikitext/Table\"
         },
         \"type\": \"move\",
         \"action\": \"move\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:37:49Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 23,
         \"ns\": 0,
         \"title\": \"Wikitext/Tables\",
         \"pageid\": 9,
         \"logpage\": 9,
         \"params\": {
           \"curid\": 22,
           \"previd\": 0,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:31:36Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 22,
         \"ns\": 0,
         \"title\": \"Wikitext\",
         \"pageid\": 7,
         \"logpage\": 7,
         \"params\": {
           \"curid\": 21,
           \"previd\": 19,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:18:38Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 21,
         \"ns\": 10,
         \"title\": \"Template:Test\",
         \"pageid\": 0,
         \"logpage\": 8,
         \"params\": {},
         \"type\": \"delete\",
         \"action\": \"delete\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:16:40Z\",
         \"comment\": \"content was: \\\"test\\\" (and the only contributor was \\\"[[Special:Contributions/JohnDoe|JohnDoe]]\\\")\"
       },
       {
         \"logid\": 20,
         \"ns\": 10,
         \"title\": \"Template:Test\",
         \"pageid\": 0,
         \"logpage\": 8,
         \"params\": {
           \"curid\": 20,
           \"previd\": 0,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:15:13Z\",
         \"comment\": \"\"
       },
       {
         \"logid\": 19,
         \"ns\": 0,
         \"title\": \"Wikitext\",
         \"pageid\": 7,
         \"logpage\": 7,
         \"params\": {
           \"curid\": 19,
           \"previd\": 18,
           \"auto\": \"\"
         },
         \"type\": \"patrol\",
         \"action\": \"patrol\",
         \"user\": \"JohnDoe\",
         \"timestamp\": \"2016-05-16T10:14:55Z\",
         \"comment\": \"\"
       }
     ]
   }
 }"
|spath
0 Karma
Reply

ryanoconnor
Builder
‎05-30-2016 04:17 PM

This looks like valid JSON according to http://jsonlint.com/

Have you tried setting up a props.conf file with the following:

[your_sourcetype_name_here]
KV_MODE = json
0 Karma
Reply
Get Updates on the Splunk Community!

Splunk Enterprise Security: Your Command Center for PCI DSS Compliance

Every security professional knows the drill. The PCI DSS audit is approaching, and suddenly everyone's asking ...

Developer Spotlight with Guilhem Marchand

From Splunk Engineer to Founder: The Journey Behind TrackMe    After spending over 12 years working full time ...

Cisco Catalyst Center Meets Splunk ITSI: From 'Payments Are Down' to Root Cause in ...

The Problem: When Networks and Services Don't Talk Payment systems fail at a retail location. Customers are ...