Splunk Cloud Platform
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Ask a Question

Json data not extracting properly

chandrasekhar46
Loves-to-Learn Everything
‎07-17-2024 03:17 AM

i am trying to upload json file using UI in Splunk cloud and applying settings for parsing as below but data is coming as a single event 

[custom_json_sourcetype]
INDEXED_EXTRACTIONS = json
SHOULD_LINEMERGE = false
KV_MODE = json
LINE_BREAKER = },\s*{

please advise correct settings to apply under sourcetypes in web when uploading

here is the data:

 

{
    "sourcetype": "testoracle_sourcetype",
    "data": {
        "cdb_tbs_check": [
            {
                "check_error": "",
                "check_name": "cdb_tbs_check",
                "check_status": "OK",
                "current_use_mb": "1355",
                "percent_used": "2",
                "tablespace_name": "SYSTEM",
                "total_physical_all_mb": "65536"
            },
            {
                "check_error": "",
                "check_name": "cdb_tbs_check",
                "check_status": "OK",
                "current_use_mb": "23596",
                "percent_used": "36",
                "tablespace_name": "SYSAUX",
                "total_physical_all_mb": "65536"
            },
            {
                "check_error": "",
                "check_name": "cdb_tbs_check",
                "check_status": "OK",
                "current_use_mb": "29",
                "percent_used": "0",
                "tablespace_name": "UNDOTBS1",
                "total_physical_all_mb": "65536"
            },
           
Labels (2)
0 Karma
Reply

PickleRick
SplunkTrust
SplunkTrust
‎07-18-2024 05:32 AM

With this data you will have some "bad events" - while you might be able to extract the structures from the middle but you will have some dangling "headers" or "footers". I'd suggest you pass this through some external filter extracting the contents based on structure, not just breaking with regex.

0 Karma
Reply

richgalloway
SplunkTrust
SplunkTrust
‎07-17-2024 05:32 AM

Don't use both INDEXED_EXTRACTIONS = JSON and KV_MODE=json in the same stanza or the fields will be extracted twice.

The LINE_BREAKER setting requires a capture group. 

Try these settings

[custom_json_sourcetype]
SHOULD_LINEMERGE = false
KV_MODE = json
LINE_BREAKER = }(,\s*){
---
If this reply helps you, Karma would be appreciated.
0 Karma
Reply

chandrasekhar46
Loves-to-Learn Everything
‎07-17-2024 10:21 PM

i have applied but data is events are getting merged in online please check attachments

 

sorry i have modified the json file and here is it what will the sourcetype settings


[
{
"sourcetype": "testoracle_sourcetype",
"check_name": "cdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "1355",
"percent_used": "2",
"tablespace_name": "SYSTEM",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "cdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "23596",
"percent_used": "36",
"tablespace_name": "SYSAUX",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "cdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "29",
"percent_used": "0",
"tablespace_name": "UNDOTBS1",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "cdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "4",
"percent_used": "0",
"tablespace_name": "USERS",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "fra_check",
"check_error": "",
"check_status": "OK",
"flash_in_gb": "40",
"flash_reclaimable_gb": "0",
"flash_used_in_gb": "1.5",
"percent_of_space_used": "3.74",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "General_parameters",
"check_error": "",
"check_status": "OK",
"database_major_version": "19",
"database_minor_version": "0",
"database_name": "C2N48617",
"database_version": "19.0.0.0.0",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617",
"script_version": "1.0"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "76",
"pdb_name": "O1S48633",
"percent_used": "0",
"tablespace_name": "UNDOTBS1",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "5",
"pdb_name": "O1S48633",
"percent_used": "0",
"tablespace_name": "TOOLS",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "21",
"pdb_name": "O1NN2467",
"percent_used": "0",
"tablespace_name": "UNDOTBS1",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "627",
"pdb_name": "O1NN2467",
"percent_used": "1",
"tablespace_name": "SYSAUX",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "784",
"pdb_name": "O1S48633",
"percent_used": "1",
"tablespace_name": "SYSTEM",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "1547",
"pdb_name": "O1NN8944",
"percent_used": "2",
"tablespace_name": "SYSAUX",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "1149",
"pdb_name": "O1S48633",
"percent_used": "2",
"tablespace_name": "USERS",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "58",
"pdb_name": "O1NN8944",
"percent_used": "0",
"tablespace_name": "UNDOTBS1",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
},
{
"sourcetype": "testoracle_sourcetype",
"check_name": "pdb_tbs_check",
"check_error": "",
"check_status": "OK",
"current_use_mb": "7804",
"pdb_name": "O1S48633",
"percent_used": "12",
"tablespace_name": "SYSAUX",
"total_physical_all_mb": "65536",
"database_name": "C2N48617",
"host_name": "flosclnrhv03.pharma.aventis.com",
"instance_name": "C2N48617"
}
]

0 Karma
Reply

richgalloway
SplunkTrust
SplunkTrust
‎07-18-2024 05:16 AM

Try this variation on the settings.  It should better account for newlines.

[custom_json_sourcetype]
SHOULD_LINEMERGE = false
KV_MODE = json
LINE_BREAKER = }(,[\S\s]*){
---
If this reply helps you, Karma would be appreciated.
0 Karma
Reply

chandrasekhar46
Loves-to-Learn Everything
‎07-17-2024 03:18 AM

EXAMPLE DATA:

 

{ "sourcetype": "testoracle_sourcetype", "data": { "cdb_tbs_check": [ { "check_error": "", "check_name": "cdb_tbs_check", "check_status": "OK", "current_use_mb": "1355", "percent_used": "2", "tablespace_name": "SYSTEM", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "cdb_tbs_check", "check_status": "OK", "current_use_mb": "23596", "percent_used": "36", "tablespace_name": "SYSAUX", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "cdb_tbs_check", "check_status": "OK", "current_use_mb": "29", "percent_used": "0", "tablespace_name": "UNDOTBS1", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "cdb_tbs_check", "check_status": "OK", "current_use_mb": "4", "percent_used": "0", "tablespace_name": "USERS", "total_physical_all_mb": "65536" } ], "fra_check": [ { "check_error": "", "check_name": "fra_check", "check_status": "OK", "flash_in_gb": "40", "flash_reclaimable_gb": "0", "flash_used_in_gb": "1.5", "percent_of_space_used": "3.74" } ], "global_parameters": { "check_error": "", "check_name": "General_parameters", "check_status": "OK", "database_major_version": "19", "database_minor_version": "0", "database_name": "C2N48617", "database_version": "19.0.0.0.0", "host_name": "flosclnrhv03.pharma.aventis.com", "instance_name": "C2N48617", "script_version": "1.0" }, "pdb_tbs_check": [ { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "76", "pdb_name": "O1S48633", "percent_used": "0", "tablespace_name": "UNDOTBS1", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "5", "pdb_name": "O1S48633", "percent_used": "0", "tablespace_name": "TOOLS", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "21", "pdb_name": "O1NN2467", "percent_used": "0", "tablespace_name": "UNDOTBS1", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "627", "pdb_name": "O1NN2467", "percent_used": "1", "tablespace_name": "SYSAUX", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "784", "pdb_name": "O1S48633", "percent_used": "1", "tablespace_name": "SYSTEM", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "1547", "pdb_name": "O1NN8944", "percent_used": "2", "tablespace_name": "SYSAUX", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "1149", "pdb_name": "O1S48633", "percent_used": "2", "tablespace_name": "USERS", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "58", "pdb_name": "O1NN8944", "percent_used": "0", "tablespace_name": "UNDOTBS1", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "7804", "pdb_name": "O1S48633", "percent_used": "12", "tablespace_name": "SYSAUX", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "1176", "pdb_name": "O1NN8944", "percent_used": "4", "tablespace_name": "USERS", "total_physical_all_mb": "32767" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "378", "pdb_name": "O1NN8944", "percent_used": "1", "tablespace_name": "INDX", "total_physical_all_mb": "32767" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "705", "pdb_name": "O1NN8944", "percent_used": "1", "tablespace_name": "SYSTEM", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "623", "pdb_name": "O1NN2467", "percent_used": "1", "tablespace_name": "SYSTEM", "total_physical_all_mb": "65536" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "3", "pdb_name": "O1S48633", "percent_used": "0", "tablespace_name": "AUDIT_TBS", "total_physical_all_mb": "8192" }, { "check_error": "", "check_name": "pdb_tbs_check", "check_status": "OK", "current_use_mb": "128", "pdb_name": "O1S48633", "percent_used": "0", "tablespace_name": "USRINDEX", "total_physical_all_mb": "65536" } ], "processes": { "check_error": "", "check_name": "processes", "check_status": "OK", "process_current_value": "294", "process_limit": "1000", "process_percent": "29.4" }, "queue_mem_check": [ { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "AQ$_Q_PIWORKTASK_TAB_E", "queue_owner": "LIVE2459_VAL", "queue_sharable_mem": "4072" }, { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "AQ$_Q_TASKREPORTWORKTASK_TAB_E", "queue_owner": "LIVE2459_VAL", "queue_sharable_mem": "4072" }, { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "AQ$_Q_LABELWORKTASK_TAB_E", "queue_owner": "LIVE2459_VAL", "queue_sharable_mem": "4072" }, { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "AQ$_Q_PIPROCESS_TAB_E", "queue_owner": "LIVE2459_VAL", "queue_sharable_mem": "4072" }, { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "AQ$_ALERT_QT_E", "queue_owner": "SYS", "queue_sharable_mem": "4032" }, { "check_error": "", "check_name": "queue_mem_check", "check_status": "OK", "queue_name": "ALERT_QUE", "queue_owner": "SYS", "queue_sharable_mem": "0" } ], "script_version": "1.0", "sessions": { "check_error": "", "check_name": "sessions", "check_status": "OK", "sessions_current_value": "293", "sessions_limit": "1536", "sessions_percent": "19.08" } } }

0 Karma
Reply
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...

Global Splunk User Group Events: May + June 2026

Your Splunk Community Awaits: Discover Upcoming User Group Events Worldwide    Staying ahead in the fast-paced ...