topic Re: How can I index JSON that's embedded in HMTL? in All Apps and Add-ons

How can I index JSON that's embedded in HMTL?

mgagliardi — Fri, 10 Feb 2017 14:31:10 GMT

I'm querying an endpoint that is returning JSON embedded in HTML. Like so:

<HTML><BODY> <pre id="info">...</pre> &lt;br/&gt; <script>document.getElementById("info").innerHTML = JSON.stringify({"perfInfo":{"processes":79,"threads":1662,"PhysicalAvailable":11717,"PhysicalTotal":16383,"CommitTotal":5856,"CommitLimit":22525,"CommitPeak":7155,"KernelNonpaged":100,"KernelPaged":209,"KernelTotal":310},"sysInfo":{"powerProfile":"7","hostname":"HOSTNAME","Version":"7.5.0.206","UDI":"(01)00853866003209(11)170203(10)7.5.0.206","BootstrapperProfile":"Enterprise - Management","hwProductName":"VMware Virtual Platform","hwBIOSVersion":"6.00","osVersion":"Windows Server 2012 (R2) (6.3 Server build 9600)"},"gpuInfo":{"numGPUs":0},"Sessions":{"available":1,"capacity":1,"assigned":[],"passive":0,"allowingNewConnections":true},"slots":{"totalSlots":1,"availableSlots":1,"passiveSlots":0,"userRamReservedMB":15559,"userRamRemainingMB":13639}}
, null, 4); </script>&lt;br/&gt;</BODY></HTML>

I want just the JSON but I'm having a hell of a time figuring out how to strip away or ignore the HTML. Could anyone offer some pointers? TIA!

Re: How can I index JSON that's embedded in HMTL?

s2_splunk — Fri, 10 Feb 2017 18:23:35 GMT

Configure this in props.conf for your sourcetype:

[your_sourcetype]
SEDCMD-removeHTMLBeginning = s/<HTML>[\s\S]*\(\{/{/g
SEDCMD-removeHTMLEnd = s/\}\}[\s\S]*$/}}/g

That worked for me with your sample response.

Re: How can I index JSON that's embedded in HMTL?

mgagliardi — Fri, 10 Feb 2017 19:21:06 GMT

FWIW your answer works, but I also found that setting an HTTP header of Accept=application/json got me where I wanted to go (via the REST API data source).

Re: How can I index JSON that's embedded in HMTL?

s2_splunk — Fri, 10 Feb 2017 19:23:05 GMT

Even better! 🙂