Getting Data In

Can not choose default sourcetype=_json while onboarding

Sandivsu
Loves-to-Learn

We can not choose default source type _json while onboarding.

Need to extract the json data within the log file, which is essential for an app owner.

log format - 2024-01-01T09:50:44+01:00 hostname APP2SAP[354]: {JSON data}

I have a splunk intermediate forwarder read these log files. Log file has non-json data followed by json data which bread n butter for application team (log format as shown above).

If I forward the data as-is to splunk, extraction is not proper, since it has non-json data at beginning.

Now, I need props n (or) transforms to extract, which I am not sure how.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Unfortunately, at the moment Splunk cannot automatically extract the structured data if it's not the whole event (as in your case - the json part is preceeded by non-json header).

There is an open idea for that https://ideas.splunk.com/ideas/EID-I-208

So far you can either parse the json part in search with help of the spath command as @VatsalJagani already showed or cut away the header part using SEDCMD or INGEST_EVAL (possibly extracting indexed fields if needed prior to removing the non-structured part).

As a side note - you should _not_ use the _json sourcetype. Define your own

VatsalJagani
SplunkTrust
SplunkTrust

@Sandivsu - Not sure if you can do that with props and transforms. But I'll provide a solution you can apply at the search query level.

index=<your-index> .....
| rex field=_raw "\s\w+\[\w+\]:\s(?<json_content>\{.*\})"
| spath input=json_content

 

I hope this helps!!! Kindly upvote if it does!!!

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...