Getting Data In

Scripted input with cURL

jedatt01
Builder

I would like to have splunk index the output of cURL but instead of seeing the entire html I'd like to be able to have splunk just index the data that a browser renders. Is there anyway to do this without some major scripting?

Tags (2)
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

You mean you want to parse the HTML and extract only selected parts of it?

0 Karma

gcoles
Communicator

In the scripted input, you could pipe the output of the curl command through a command like hxnormalize, which is part of the html-xml-utils package in Linux.

0 Karma

jedatt01
Builder

gcoles, this answer didn't quite work for me, however what did work was that I found out how the page was getting the dynamic information. It was using a .cgi call. So, I just created a scripted input that calls the .cgi and formats the output to key value pairs

0 Karma
Get Updates on the Splunk Community!

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

What's New in Splunk Observability - July 2025

What’s New?  We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what ...