Getting Data In

Can not choose default sourcetype=_json while onboarding

Sandivsu
Loves-to-Learn

We can not choose default source type _json while onboarding.

Need to extract the json data within the log file, which is essential for an app owner.

log format - 2024-01-01T09:50:44+01:00 hostname APP2SAP[354]: {JSON data}

I have a splunk intermediate forwarder read these log files. Log file has non-json data followed by json data which bread n butter for application team (log format as shown above).

If I forward the data as-is to splunk, extraction is not proper, since it has non-json data at beginning.

Now, I need props n (or) transforms to extract, which I am not sure how.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Unfortunately, at the moment Splunk cannot automatically extract the structured data if it's not the whole event (as in your case - the json part is preceeded by non-json header).

There is an open idea for that https://ideas.splunk.com/ideas/EID-I-208

So far you can either parse the json part in search with help of the spath command as @VatsalJagani already showed or cut away the header part using SEDCMD or INGEST_EVAL (possibly extracting indexed fields if needed prior to removing the non-structured part).

As a side note - you should _not_ use the _json sourcetype. Define your own

VatsalJagani
SplunkTrust
SplunkTrust

@Sandivsu - Not sure if you can do that with props and transforms. But I'll provide a solution you can apply at the search query level.

index=<your-index> .....
| rex field=_raw "\s\w+\[\w+\]:\s(?<json_content>\{.*\})"
| spath input=json_content

 

I hope this helps!!! Kindly upvote if it does!!!

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...