Getting Data In

Collect command to index in JSON format

alvaromari83
Path Finder

Hello!

I'm trying to generate some summarized data by using the collect command in my SPL queries.

The event format by default is

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2

Some of this fields, are actually JSON objects, like:

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2 field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"

So, to avois a mixture of formats in my collected event, I would like to index the results, using the collect command, BUT in JSON format for the whole event, something like:

{"time":"05/20/2019 07:15:00 +0200", "info_min_time"=1.000, "info_max_time":1558557310.000, "info_search_time":1558557310.410, results: {"field1":"value1" "field2":"value2", field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"...}}

Is such thing feasible? Some props tweaking?

Thank you in advance!

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@alvaromari83

Can you please try | mkjson command from JSON Tools Splunk App ?

https://splunkbase.splunk.com/app/3540/#/overview

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust
0 Karma

koshyk
Super Champion

You can look into it from two perspective
1. Is the source data the most important thing in your organisation?
2. If the data format is important

In many organisation (1) is more important. This is to preserve the original data. Splunk doesn't have any issue in extracting data even if the JSON format is mixed. You index "original" data, but just mention the json part to extract automatically

Please see this thread on how to index json mixed format. https://answers.splunk.com/answers/117121/extract-json-data-within-the-logs-json-mixed-with.html

You could even add a stanza just to extract pure JSON if need be at search time. This way you preserve the original data as well as you get key-value at search-time

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...