Getting Data In
Highlighted

how to break the json data into events?

Explorer

Hello experts,below is the json data

{"actions": [{"date": "2012-05-17 00:00:00", "action": "Read for the first time and referred to the Senate committee on Judiciary", "+amendment": null, "type": ["bill:reading:1", "committee:referred"], "actor": "upper"}, {"date": "2012-05-18 00:00:00", "action": "Read for the second time and placed on the calendar", "+amendment": null, "type": ["bill:reading:2"], "actor": "upper"}, {"date": "2012-05-21 00:00:00", "action": "Unfinished Business", "+amendment": null, "type": ["other"], "actor": "upper"}..................................

I want to break the above data starting with a date into single events,Is that possible

I tried using sourcetype=json and kv_mode=json but still it is not breaking .Please help

Tags (2)
0 Karma
Highlighted

Re: how to break the json data into events?

Explorer

i want date,action,type,bill,commite,actor in one event

0 Karma
Highlighted

Re: how to break the json data into events?

Champion

You can use this in props.conf to break your data into events starting with your date field:

LINE_BREAKER = (..){"date"

You should try it out during the "Add Data" wizard to see if everything is as you want it to be.

0 Karma
Highlighted

Re: how to break the json data into events?

Explorer

Hey Jeff thanks for the response

(..) is this also needed to be added

0 Karma
Highlighted

Re: how to break the json data into events?

Explorer

I tried doing that...In the add data..set source type...but it is not showing only as a single event

0 Karma
Highlighted

Re: how to break the json data into events?

Champion

(..) will tell splunk to take away the two characters before the opening curly bracket {. In your data, that is either a space and an opening bracket [, or a comma and a space. From props.conf:

* The regex must contain a capturing group -- a pair of parentheses which defines an identified subcomponent of the match.
* Wherever the regex matches, Splunk considers the start of the first capturing group to be the end of the previous event, and considers the end of the first capturing group to be the start of the next event.
* The contents of the first capturing group are discarded, and will not be present in any event.  You are telling Splunk that this text comes between lines.

These settings work for me, but I've encountered some strange behavior with the "Add Data" wizard from time to time. I'd suggest you set up and save your settings for the source type from the source type manager (Settings - Source Types) and select that saved source type during the "Add Data" wizard, that should make everything apply correctly even during that wizard.

For completeness' sake, here are the settings you should definitely have for your sourcetype:

LINE_BREAKER = (..){"date"
SHOULD_LINEMERGE = false
0 Karma
Highlighted

Re: how to break the json data into events?

Explorer

well I tried using your settings but the whole data is coming as one single event instead of breaking into multiple events.

0 Karma
Highlighted

Re: how to break the json data into events?

Champion

That's strange. If I paste your sample from above in a text document and try to index it with the above settings, they work. I'm on 6.3.1, but I doubt version will make a difference here.

Maybe there are other characters in the original data?

0 Karma
Highlighted

Re: how to break the json data into events?

Explorer

Name Value

CHARSET=UTF-8

INDEXED_EXTRACTIONS=json

KV_MODE=JSON

SHOULD_LINEMERGE=false

category=Structured

description=JavaScript Object Notation format. For more information, visit http://json.org/

disabled=false

pulldown_type=true

LINE_BREAKER= (..){"date"

the above is the setting I have done,is that fine or any changes needed

0 Karma
Highlighted

Re: how to break the json data into events?

Champion

Almost. You don't need KVMODE when you set INDEXEDEXTRACTIONS - the first one is for search time field extraction, which you don't need because you already have indexed extractions. Therefore, you should set KV_MODE = none.

0 Karma