Join the Conversation
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Field Extraction from complex JSON file
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I have complex JSON events ingested as *.log files. I have issues (or couldn't do) with extracting fields from this files/events. Any help on how to extract Key-Value pairs from these events would be highly appreciated. One sample event is given below. Thank you so much.
2022-07-15 12:44:03 - {
"type" : "TEST",
"r/o" : false,
"booting" : false,
"version" : "6.2.7.TS",
"user" : "DS",
"domainUUID" : null,
"access" : "NATIVE",
"remote-address" : "localhost",
"success" : true,
"ops" : [{
"address" : [
{
"subsystem" : "datasources"
},
{
"data-source" : "mode_tp"
}
],
"address" : [
{
"cservice" : "management"
},
{
"access" : "identity"
}
],
"DSdomain" : "TESTDomain"
},
{
"address" : [
{
"cservice" : "management"
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "finit"
},
{
"bucket" : "TEST"
},
{
"clocal" : "passivation"
},
{
"store" : "file"
}
],
"passivation" : true,
"purge" : false
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "finit"
},
{
"bucket" : "TEST"
}
],
"module" : "dshibernate"
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "finit"
},
{
"bucket" : "hibernate"
},
{
"clocal" : "entity"
}
]
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "finit"
},
{
"bucket" : "hibernate"
},
{
"clocal" : "entity"
},
{
"component" : "transaction"
}
],
"model" : "DSTEST"
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "infit"
},
{
"bucket" : "hibernate"
},
{
"clocal" : "entity"
},
{
"memory" : "object"
}
],
"size" : 210000
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "DS"
},
{
"workplace" : "default"
},
{
"running-spin" : "default"
}
],
"Test-threads" : 45,
"queue-length" : 60,
"max-threads" : 70,
"keepalive-time" : {
"time" : 20,
"unit" : "SECONDS"
}
},
{
"operation" : "add",
"address" : [
{
"subsystem" : "DS"
},
{
"workplace" : "default"
},
{
"long-running-threads" : "default"
}
],
"Test-threads" : 45,
"queue-length" : 70,
"max-threads" : 70,
"keepalive-time" : {
"time" : 20,
"unit" : "SECONDS"
}
},
}]
}
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @SplunkDash,
I hint to try to use spath, because regexes is a very hard way to extract fields, infact you have to create many extractions for each field, e.g. the following:
| rex "\"workplace\" : \"(?<workplace>[^\"]+)\""Ciao.
Giuseppe
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @SplunkDash,
you don't ned to extract inline, you could also extract and save extraction, but try to use spath.
Ciao.
Giuseppe
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @gcusello,
How can I save the extraction for later use without defining it as inline or transformation or macro?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you so much again. But from the web interface (UI), I can see only 2 ways we can save field extraction INLINE and Transformations. If I use +Extract New Fields or Extract Fields, it goes to INLINE (regular expression) or Field Transformations (Delimiters) option. No where is allowed me to use spath. Is there anything I am missing? Thank you so much again.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @gcusello,
Thank you so much again. But from Web interface (UI), I can see only 2 ways we can save field extraction INLINE or Transformations. If I use the option at the bottom of the left most column (where fields are listing) "+Extract New Fields", still not letting me to use spath option like as follow. Is there anything I am missing here? Please guide me if possible. Thank you!
| rex field=_raw "\d+-\d+-\d+ \d+:\d+:\d+ - (?<_raw>[\S\s]+)" | spath
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
SOC4Kafka - New Kafka Connector Powered by OpenTelemetry
Rounding off the Splunk Dashboard Contest
A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure
