Getting Data In

How do I split an array in a .json file into multiple new events?

dgutekunst
Engager

Right now I have a json file that's formatted like:

{
    "Log Files":[
        {"Date":"2014-07-18 21:22:51", "Available Bytes(kb)":3960078, ...},
        {"Date":"2014-07-18 21:24:01", "Available Bytes(kb)":4001231, ...},
        {"Date":"2014-07-18 21:25:14", "Available Bytes(kb)":3872959, ...}]}

Right now it's showing up in Splunk as:

timestamp                Date                Available Bytes(kb)
2014-07-18 21:22:51:000  2014-07-18 21:22:51 3960078
                         2014-07-18 21:24:01 4001231
                         2014-07-18 21:25:14 3872959

How can I split these up into individual events when I load the data? I can get the timestamp to correctly match the Date field, but it will still only give one date for the whole file, even though there are several lines that are each individual logs.

Tags (3)

vasanthmss
Motivator

Try something like this,

<your base search...> | table  timestamp, Date, Available | eval temp=mvzip(timestamp, mvzip(Date, Available,"###"), "###") | mvexpand temp | rex field=temp "(?<timestamp>.*)###(?<Date>.*)###(?<Available>.*)" | fields - temp

I have used the sample fields, you can try with your actual fields. Concept here is, you need to zip it with a delimiter (here ###) and expand it and extract it. This do the magic

V

s2_splunk
Splunk Employee
Splunk Employee

Is there anything in the envelope of this array that you want to keep or are you just interested in keeping the events inside the "Log Files" array?
If the latter, setup props/transforms for your sourcetype to:

  • Get rid of the root '{' line
  • Get rid of the array root line that contains "Log Files"
  • Use SEDCMD to change "}," to "}" at the end of the lines
  • Use SEDCMD to remove "]}"
  • Set TIMESTAMP_PREFIX to \{\"Date\":\"

That should result in individual, valid JSON events that should render fine in the UI.

0 Karma

pwmcintyre
Explorer

When you say "... when I load the data" do you mean at search time or index time?

0 Karma
Get Updates on the Splunk Community!

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...