how to break the json data into events?

[chaseto]

Hello experts,below is the json data

{"actions": [{"date": "2012-05-17 00:00:00", "action": "Read for the first time and referred to the Senate committee on Judiciary", "+amendment": null, "type": ["bill:reading:1", "committee:referred"], "actor": "upper"}, {"date": "2012-05-18 00:00:00", "action": "Read for the second time and placed on the calendar", "+amendment": null, "type": ["bill:reading:2"], "actor": "upper"}, {"date": "2012-05-21 00:00:00", "action": "Unfinished Business", "+amendment": null, "type": ["other"], "actor": "upper"}..................................

I want to break the above data starting with a date into single events,Is that possible

I tried using sourcetype=json and kv_mode=json but still it is not breaking .Please help

[vrmandadi]

in splunk web when you onboard the file you will have option for selecting the sourcetype...i think in web structured section you have _json...if you select that particular type splunk automatically breaks the events

[jeffland]

You can use this in props.conf to break your data into events starting with your date field:

LINE_BREAKER = (..){"date"

You should try it out during the "Add Data" wizard to see if everything is as you want it to be.

[chaseto]

Hey Jeff thanks for the response

(..) is this also needed to be added

[jeffland]

(..) will tell splunk to take away the two characters before the opening curly bracket {. In your data, that is either a space and an opening bracket [, or a comma and a space. From props.conf:

* The regex must contain a capturing group -- a pair of parentheses which defines an identified subcomponent of the match.
* Wherever the regex matches, Splunk considers the start of the first capturing group to be the end of the previous event, and considers the end of the first capturing group to be the start of the next event.
* The contents of the first capturing group are discarded, and will not be present in any event.  You are telling Splunk that this text comes between lines.

These settings work for me, but I've encountered some strange behavior with the "Add Data" wizard from time to time. I'd suggest you set up and save your settings for the source type from the source type manager (Settings - Source Types) and select that saved source type during the "Add Data" wizard, that should make everything apply correctly even during that wizard.

For completeness' sake, here are the settings you should definitely have for your sourcetype:

LINE_BREAKER = (..){"date"
SHOULD_LINEMERGE = false

[chaseto]

well I tried using your settings but the whole data is coming as one single event instead of breaking into multiple events.

[jeffland]

That's strange. If I paste your sample from above in a text document and try to index it with the above settings, they work. I'm on 6.3.1, but I doubt version will make a difference here.

Maybe there are other characters in the original data?

[chaseto]

Name Value

CHARSET=UTF-8

INDEXED_EXTRACTIONS=json

KV_MODE=JSON

SHOULD_LINEMERGE=false

category=Structured

description=JavaScript Object Notation format. For more information, visit http://json.org/

disabled=false

pulldown_type=true

LINE_BREAKER= (..){"date"

the above is the setting I have done,is that fine or any changes needed

[jeffland]

Almost. You don't need KV_MODE when you set INDEXED_EXTRACTIONS - the first one is for search time field extraction, which you don't need because you already have indexed extractions. Therefore, you should set KV_MODE = none.

[chaseto]

I have tried that too,but still the event summary still shows only one event

Event Summary

Sample size 10000 bytes
Events 1
Event time distribution
0
1
5/17/12 12:00 AM: 1
5/17/12 12:00 AM5/17/12 12:00 AM
Number of lines per event

of lines # of events % of events

1 1

[jeffland]

Hm, I just noticed you have a space before the line breaker regex - if you remove that, restart your splunk and try to add the data once more, I don't know why it shouldn't work. It does on my machine.

[chaseto]

I tried doing that...In the add data..set source type...but it is not showing only as a single event

[chaseto]

i want date,action,type,bill,commite,actor in one event