Getting Data In

how to break the json data into events?

chaseto
Explorer

Hello experts,below is the json data

{"actions": [{"date": "2012-05-17 00:00:00", "action": "Read for the first time and referred to the Senate committee on Judiciary", "+amendment": null, "type": ["bill:reading:1", "committee:referred"], "actor": "upper"}, {"date": "2012-05-18 00:00:00", "action": "Read for the second time and placed on the calendar", "+amendment": null, "type": ["bill:reading:2"], "actor": "upper"}, {"date": "2012-05-21 00:00:00", "action": "Unfinished Business", "+amendment": null, "type": ["other"], "actor": "upper"}..................................

I want to break the above data starting with a date into single events,Is that possible

I tried using sourcetype=json and kv_mode=json but still it is not breaking .Please help

Tags (2)
0 Karma

vrmandadi
Builder

in splunk web when you onboard the file you will have option for selecting the sourcetype...i think in web structured section you have _json...if you select that particular type splunk automatically breaks the events

0 Karma

jeffland
SplunkTrust
SplunkTrust

You can use this in props.conf to break your data into events starting with your date field:

LINE_BREAKER = (..){"date"

You should try it out during the "Add Data" wizard to see if everything is as you want it to be.

0 Karma

chaseto
Explorer

Hey Jeff thanks for the response

(..) is this also needed to be added

0 Karma

jeffland
SplunkTrust
SplunkTrust

(..) will tell splunk to take away the two characters before the opening curly bracket {. In your data, that is either a space and an opening bracket [, or a comma and a space. From props.conf:

* The regex must contain a capturing group -- a pair of parentheses which defines an identified subcomponent of the match.
* Wherever the regex matches, Splunk considers the start of the first capturing group to be the end of the previous event, and considers the end of the first capturing group to be the start of the next event.
* The contents of the first capturing group are discarded, and will not be present in any event.  You are telling Splunk that this text comes between lines.

These settings work for me, but I've encountered some strange behavior with the "Add Data" wizard from time to time. I'd suggest you set up and save your settings for the source type from the source type manager (Settings - Source Types) and select that saved source type during the "Add Data" wizard, that should make everything apply correctly even during that wizard.

For completeness' sake, here are the settings you should definitely have for your sourcetype:

LINE_BREAKER = (..){"date"
SHOULD_LINEMERGE = false
0 Karma

chaseto
Explorer

well I tried using your settings but the whole data is coming as one single event instead of breaking into multiple events.

0 Karma

jeffland
SplunkTrust
SplunkTrust

That's strange. If I paste your sample from above in a text document and try to index it with the above settings, they work. I'm on 6.3.1, but I doubt version will make a difference here.

Maybe there are other characters in the original data?

0 Karma

chaseto
Explorer

Name Value

CHARSET=UTF-8

INDEXED_EXTRACTIONS=json

KV_MODE=JSON

SHOULD_LINEMERGE=false

category=Structured

description=JavaScript Object Notation format. For more information, visit http://json.org/

disabled=false

pulldown_type=true

LINE_BREAKER= (..){"date"

the above is the setting I have done,is that fine or any changes needed

0 Karma

jeffland
SplunkTrust
SplunkTrust

Almost. You don't need KV_MODE when you set INDEXED_EXTRACTIONS - the first one is for search time field extraction, which you don't need because you already have indexed extractions. Therefore, you should set KV_MODE = none.

0 Karma

chaseto
Explorer

I have tried that too,but still the event summary still shows only one event

Event Summary

Sample size 10000 bytes
Events 1
Event time distribution
0
1
5/17/12 12:00 AM: 1
5/17/12 12:00 AM5/17/12 12:00 AM
Number of lines per event

of lines # of events % of events

1 1

0 Karma

jeffland
SplunkTrust
SplunkTrust

Hm, I just noticed you have a space before the line breaker regex - if you remove that, restart your splunk and try to add the data once more, I don't know why it shouldn't work. It does on my machine.

0 Karma

chaseto
Explorer

I tried doing that...In the add data..set source type...but it is not showing only as a single event

0 Karma

chaseto
Explorer

i want date,action,type,bill,commite,actor in one event

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...