Getting Data In

Ingesting JSON formatted logs into Splunk

lball
Explorer

I'm able to get JSON formatted linux os & modx web logs into a Splunk index, but they are not formatted or parsed. How can I get the logs to be efficiently parsed into the index so that they can be searched and used for reporting & dashboards. If this is impractical, is there a better way to get modx web logs into Splunk? If I am able to get them sent in syslog format will they parse correctly?

Tags (2)
0 Karma

jeffbat
Path Finder

If you can grab a copy of the file you are trying to read, then on a dev splunk instance walk through the Add Data function in the web console.

Just import your file directly and when at the Set Source Type, choose, Structured->_json

You can then make sure it looks like it is parsing correctly and do a Save As to a new name/sourcetype name. Then when you finish getting it all read in, you can go to your drive and look for the inputs/props/transforms conf files it would create. Then you can use those on the forwarder you are trying to read the file originally from (or pushed out through a deployment server in an app).

0 Karma

hookupgeek
New Member

Thanks for the tip!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the props.conf settings for that sourcetype?

---
If this reply helps you, Karma would be appreciated.
0 Karma

worshamn
Contributor

Like richgalloway mentioned in props.conf, make sure it has set KV_MODE = json. Also make sure that each event is a complete JSON event (for example doesn't have any text written before the JSON)

You could always copy a JSON line and paste it into a JSON pretty print web site to make sure they can parse it, like https://jsonformatter.org/json-pretty-print.

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...