Getting Data In

What will the JSON Path for breaking this JSON be while ingesting it in Splunk ?

beingkaran
New Member

Please note that I want the JSON path expression and want to break this before ingesting it splunk and not to use spath after ingesting

JSON
[
{
"ticket_id":"423535",
},
{
"ticket_id":"422946",
},
{
"ticket_id":"272791",
},
{
"ticket_id":"240391",
},
]

Break it as

Event1:
{
"ticket_id":"423535",
}
Event 2:
{
"ticket_id":"422946",
}
Event 3:
{
"ticket_id":"272791",
}
Event 4:
{
"ticket_id":"240391",
}

Tags (2)
0 Karma
1 Solution

spavin
Path Finder

Hi @beingkaran,

If you on-board the JSON as-is, you can create search-time extractions to get all the fields, then you won't need to use spath to use them.

E.G. the following props.conf:

[ANSWERS_SEARCHTIME]
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
KV_MODE=JSON
LINE_BREAKER=\}(,?)
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
SEDCMD-firstbracket=s/^\[//
SEDCMD-lastbracket=s/\]$//
EXTRACT-ticket_id = "ticket_id"\s*:\s*"([^"]+)"
EXTRACT-circuit_id = "circuit_id"\s*:\s*"([^"]+)"
EXTRACT-customer_id = "customer_id"\s*:\s*"([^"]+)"
EXTRACT-prob_type = "prob_type"\s*:\s*"([^"]+)"
EXTRACT-severity = "severity"\s*:\s*"([^"]+)"

That will break the event up (assuming no nested brackets) and create search-time fields for all the keys. The props.conf will need to go on both the search head (for search-time extractions) and indexer (for the rest of the props config).

alt text

An alternative is to use indexed extractions. You can take advantage of the INDEXED_EXTRACTIONS setting:

[SPLUNK_ANSWERS]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = json
LINE_BREAKER = ([\r\n]+)

This results in all the fields being available, with no spath required:

alt text

Note that indexed extractions will take more space on indexers, and will need more CPU to initially on-board. You get the ability to use tstats though, which you can't do with search-time extractions.

View solution in original post

0 Karma

spavin
Path Finder

Hi @beingkaran,

If you on-board the JSON as-is, you can create search-time extractions to get all the fields, then you won't need to use spath to use them.

E.G. the following props.conf:

[ANSWERS_SEARCHTIME]
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
KV_MODE=JSON
LINE_BREAKER=\}(,?)
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
SEDCMD-firstbracket=s/^\[//
SEDCMD-lastbracket=s/\]$//
EXTRACT-ticket_id = "ticket_id"\s*:\s*"([^"]+)"
EXTRACT-circuit_id = "circuit_id"\s*:\s*"([^"]+)"
EXTRACT-customer_id = "customer_id"\s*:\s*"([^"]+)"
EXTRACT-prob_type = "prob_type"\s*:\s*"([^"]+)"
EXTRACT-severity = "severity"\s*:\s*"([^"]+)"

That will break the event up (assuming no nested brackets) and create search-time fields for all the keys. The props.conf will need to go on both the search head (for search-time extractions) and indexer (for the rest of the props config).

alt text

An alternative is to use indexed extractions. You can take advantage of the INDEXED_EXTRACTIONS setting:

[SPLUNK_ANSWERS]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = json
LINE_BREAKER = ([\r\n]+)

This results in all the fields being available, with no spath required:

alt text

Note that indexed extractions will take more space on indexers, and will need more CPU to initially on-board. You get the ability to use tstats though, which you can't do with search-time extractions.

0 Karma

to4kawa
Ultra Champion

KV_MODE=JSON

 EXTRACT-ticket_id = "ticket_id"\s*:\s*"([^"]+)"
 EXTRACT-circuit_id = "circuit_id"\s*:\s*"([^"]+)"
 EXTRACT-customer_id = "customer_id"\s*:\s*"([^"]+)"
 EXTRACT-prob_type = "prob_type"\s*:\s*"([^"]+)"
 EXTRACT-severity = "severity"\s*:\s*"([^"]+)"

Do you need above?

0 Karma

spavin
Path Finder

You can do either or both, depending on how you want the extracted fields to look.

KV_MODE will give extractions in the form of: {}.fieldname

EXTRACT- will give extractions in the form of: fieldname

0 Karma

to4kawa
Ultra Champion

Thanks @spavin

JSON_TRIM_BRACES_IN_ARRAY_NAMES = <boolean>
* Whether or not the JSON parser for 'INDEXED_EXTRACTIONS' strips curly
  braces from names of fields that are defined as arrays in JSON events.
* When the JSON parser extracts fields from JSON events, by default, it
  extracts array field names with the curly braces that indicate they
  are arrays ("{}") intact.
* For example, given the following partial JSON event:
    {"datetime":"08-20-2015 10:32:25.267 -0700","log_level":"INFO",...,
      data:{...,"fs_type":"ext4","mount_point":["/disk48","/disk22"],...}}

  Because the "mount_point" field in this event is an array of two
  values ("/disk48" and "/disk22"), the JSON parser sees the field as an
  array, and extracts it as such, including the braces that identify
  it as an array. The resulting field name is "data.mount_point{}").
* Set 'JSON_TRIM_BRACES_IN_ARRAY_NAMES' to "true" if you want the JSON
  parser to strip these curly braces from array field names. (In this
  example, the resulting field is instead "data.mount_point").
* CAUTION: Setting this to "true" makes array field names that are extracted
  at index time through the JSON parser inconsistent with search-time
  extraction of array field names through the 'spath' search command.
* Default: false

this option is useful, too.

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@beingkaran

Is it actual event you are getting ? if not then can you please share original event with masking confidential information.

0 Karma

beingkaran
New Member

Hi kamlesh this is the gist of it I can disclose more due to customer data but the structure is exact now I want to know the json path that will break them ticket by ticket

[
{
"ticket_id": "EN_3",
"circuit_id": "ENT3",
"customer_id": "C-1",
"prob_type": "LINKDOWN",
"severity": "Critical"
},
{
"ticket_id": "EN_4",
"circuit_id": "ENT4",
"customer_id": "C-2",
"prob_type": "LINKDOWN",
"severity": "Critical"
}
]

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...