Getting Data In

Collect command to index in JSON format

alvaromari83
Path Finder

Hello!

I'm trying to generate some summarized data by using the collect command in my SPL queries.

The event format by default is

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2

Some of this fields, are actually JSON objects, like:

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2 field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"

So, to avois a mixture of formats in my collected event, I would like to index the results, using the collect command, BUT in JSON format for the whole event, something like:

{"time":"05/20/2019 07:15:00 +0200", "info_min_time"=1.000, "info_max_time":1558557310.000, "info_search_time":1558557310.410, results: {"field1":"value1" "field2":"value2", field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"...}}

Is such thing feasible? Some props tweaking?

Thank you in advance!

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@alvaromari83

Can you please try | mkjson command from JSON Tools Splunk App ?

https://splunkbase.splunk.com/app/3540/#/overview

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust
0 Karma

koshyk
Super Champion

You can look into it from two perspective
1. Is the source data the most important thing in your organisation?
2. If the data format is important

In many organisation (1) is more important. This is to preserve the original data. Splunk doesn't have any issue in extracting data even if the JSON format is mixed. You index "original" data, but just mention the json part to extract automatically

Please see this thread on how to index json mixed format. https://answers.splunk.com/answers/117121/extract-json-data-within-the-logs-json-mixed-with.html

You could even add a stanza just to extract pure JSON if need be at search time. This way you preserve the original data as well as you get key-value at search-time

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...