Getting Data In

How to configure props.conf to break JSON into events and get the correct timestamp?

Communicator

Hi all.

I'm getting some JSON files from API figures.
This is what I receive:

  "2015-08-02": {
    "downloads": 49,
    "updates": 4,
    "returns": 0,
    "net_downloads": 49,
    "promos": 0,
    "revenue": "54.98",
    "edu_downloads": 0,
    "gifts": 0,
    "gift_redemptions": 0,
    "date": "2015-08-02"
  },
  "2015-08-03": {
    "downloads": 41,
    "updates": 6,
    "returns": 0,
    "net_downloads": 41,
    "promos": 0,
    "revenue": "0.00",
    "edu_downloads": 0,
    "gifts": 0,
    "gift_redemptions": 0,
    "date": "2015-08-03"
  },
  "2015-08-04": {
    "downloads": 71,
    "updates": 5,
    "returns": 0,
    "net_downloads": 71,
    "promos": 0,
    "revenue": "53.07",
    "edu_downloads": 0,
    "gifts": 0,
    "gift_redemptions": 0,
    "date": "2015-08-04"
  }

I'm trying to create a sourcetype for this (Data input -> Set Sourcetype) since _json did not work, but no success so far. I was not able to break it into events and get the correct timestamp.

How can I configure the props.conf so I can brake the events and get the timestamp?

Thanks is advance.

0 Karma
1 Solution

Champion

Your JSON data is not well formatted as it looks like you have multiple events in a single json object. For json settings to work your data must look something like whats below foreach event:

{"2015-08-02": {
     "downloads": 49,
     "updates": 4,
     "returns": 0,
     "net_downloads": 49,
     "promos": 0,
     "revenue": "54.98",
     "edu_downloads": 0,
     "gifts": 0,
     "gift_redemptions": 0,
     "date": "2015-08-02"
   }
}

The way you appear to be breaking data your event looks something like what below which is not proper json:

"2015-08-02": {
     "downloads": 49,
     "updates": 4,
     "returns": 0,
     "net_downloads": 49,
     "promos": 0,
     "revenue": "54.98",
     "edu_downloads": 0,
     "gifts": 0,
     "gift_redemptions": 0,
     "date": "2015-08-02"
   }

You could write a script to encapsulate each event in {}.

If you just care about breaking the event correctly and not about it being in proper json you could use the following

[kindofJSON]
 BREAK_ONLY_BEFORE="\d{4}-\d{2}-\d{2}":

View solution in original post

0 Karma

Champion

Your JSON data is not well formatted as it looks like you have multiple events in a single json object. For json settings to work your data must look something like whats below foreach event:

{"2015-08-02": {
     "downloads": 49,
     "updates": 4,
     "returns": 0,
     "net_downloads": 49,
     "promos": 0,
     "revenue": "54.98",
     "edu_downloads": 0,
     "gifts": 0,
     "gift_redemptions": 0,
     "date": "2015-08-02"
   }
}

The way you appear to be breaking data your event looks something like what below which is not proper json:

"2015-08-02": {
     "downloads": 49,
     "updates": 4,
     "returns": 0,
     "net_downloads": 49,
     "promos": 0,
     "revenue": "54.98",
     "edu_downloads": 0,
     "gifts": 0,
     "gift_redemptions": 0,
     "date": "2015-08-02"
   }

You could write a script to encapsulate each event in {}.

If you just care about breaking the event correctly and not about it being in proper json you could use the following

[kindofJSON]
 BREAK_ONLY_BEFORE="\d{4}-\d{2}-\d{2}":

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Try using break only before, or break only after... and time prefix:

[jsonsourcetype]
BREAK_ONLY_BEFORE=},|}
TIME_PREFIX = "date":
0 Karma