Solved: Re: How can I index JSON that's embedded in HMTL?

mgagliardi · ‎02-10-2017

I'm querying an endpoint that is returning JSON embedded in HTML. Like so:

<HTML><BODY> <pre id="info">...</pre> &lt;br/&gt; <script>document.getElementById("info").innerHTML = JSON.stringify({"perfInfo":{"processes":79,"threads":1662,"PhysicalAvailable":11717,"PhysicalTotal":16383,"CommitTotal":5856,"CommitLimit":22525,"CommitPeak":7155,"KernelNonpaged":100,"KernelPaged":209,"KernelTotal":310},"sysInfo":{"powerProfile":"7","hostname":"HOSTNAME","Version":"7.5.0.206","UDI":"(01)00853866003209(11)170203(10)7.5.0.206","BootstrapperProfile":"Enterprise - Management","hwProductName":"VMware Virtual Platform","hwBIOSVersion":"6.00","osVersion":"Windows Server 2012 (R2) (6.3 Server build 9600)"},"gpuInfo":{"numGPUs":0},"Sessions":{"available":1,"capacity":1,"assigned":[],"passive":0,"allowingNewConnections":true},"slots":{"totalSlots":1,"availableSlots":1,"passiveSlots":0,"userRamReservedMB":15559,"userRamRemainingMB":13639}}
, null, 4); </script>&lt;br/&gt;</BODY></HTML>

I want just the JSON but I'm having a hell of a time figuring out how to strip away or ignore the HTML. Could anyone offer some pointers? TIA!

s2_splunk · ‎02-10-2017

Configure this in props.conf for your sourcetype:

[your_sourcetype]
SEDCMD-removeHTMLBeginning = s/<HTML>[\s\S]*\(\{/{/g
SEDCMD-removeHTMLEnd = s/\}\}[\s\S]*$/}}/g

That worked for me with your sample response.

View solution in original post

s2_splunk · ‎02-10-2017

Configure this in props.conf for your sourcetype:

[your_sourcetype]
SEDCMD-removeHTMLBeginning = s/<HTML>[\s\S]*\(\{/{/g
SEDCMD-removeHTMLEnd = s/\}\}[\s\S]*$/}}/g

That worked for me with your sample response.

mgagliardi · ‎02-10-2017

FWIW your answer works, but I also found that setting an HTTP header of Accept=application/json got me where I wanted to go (via the REST API data source).

s2_splunk · ‎02-10-2017

Even better! 🙂

How can I index JSON that's embedded in HMTL?

Announcing Scheduled Export GA for Dashboard Studio

Extending Observability Content to Splunk Cloud

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!