Getting Data In

Scripted input with cURL

jedatt01
Builder

I would like to have splunk index the output of cURL but instead of seeing the entire html I'd like to be able to have splunk just index the data that a browser renders. Is there anyway to do this without some major scripting?

Tags (2)
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

You mean you want to parse the HTML and extract only selected parts of it?

0 Karma

gcoles
Communicator

In the scripted input, you could pipe the output of the curl command through a command like hxnormalize, which is part of the html-xml-utils package in Linux.

0 Karma

jedatt01
Builder

gcoles, this answer didn't quite work for me, however what did work was that I found out how the page was getting the dynamic information. It was using a .cgi call. So, I just created a scripted input that calls the .cgi and formats the output to key value pairs

0 Karma
Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...