Getting Data In

Ingesting JSON formatted logs into Splunk

lball
Explorer

I'm able to get JSON formatted linux os & modx web logs into a Splunk index, but they are not formatted or parsed. How can I get the logs to be efficiently parsed into the index so that they can be searched and used for reporting & dashboards. If this is impractical, is there a better way to get modx web logs into Splunk? If I am able to get them sent in syslog format will they parse correctly?

Tags (2)
0 Karma

jeffbat
Path Finder

If you can grab a copy of the file you are trying to read, then on a dev splunk instance walk through the Add Data function in the web console.

Just import your file directly and when at the Set Source Type, choose, Structured->_json

You can then make sure it looks like it is parsing correctly and do a Save As to a new name/sourcetype name. Then when you finish getting it all read in, you can go to your drive and look for the inputs/props/transforms conf files it would create. Then you can use those on the forwarder you are trying to read the file originally from (or pushed out through a deployment server in an app).

0 Karma

hookupgeek
New Member

Thanks for the tip!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the props.conf settings for that sourcetype?

---
If this reply helps you, Karma would be appreciated.
0 Karma

worshamn
Contributor

Like richgalloway mentioned in props.conf, make sure it has set KV_MODE = json. Also make sure that each event is a complete JSON event (for example doesn't have any text written before the JSON)

You could always copy a JSON line and paste it into a JSON pretty print web site to make sure they can parse it, like https://jsonformatter.org/json-pretty-print.

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...