Getting Data In

Scripted input with cURL

jedatt01
Builder

I would like to have splunk index the output of cURL but instead of seeing the entire html I'd like to be able to have splunk just index the data that a browser renders. Is there anyway to do this without some major scripting?

Tags (2)
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

You mean you want to parse the HTML and extract only selected parts of it?

0 Karma

gcoles
Communicator

In the scripted input, you could pipe the output of the curl command through a command like hxnormalize, which is part of the html-xml-utils package in Linux.

0 Karma

jedatt01
Builder

gcoles, this answer didn't quite work for me, however what did work was that I found out how the page was getting the dynamic information. It was using a .cgi call. So, I just created a scripted input that calls the .cgi and formats the output to key value pairs

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...