Dashboards & Visualizations

How do I deal with a log that contains HTML

JShood
Explorer

We have a log that contains an HTML formatted dump of memory variables at the time an error occurred.

Is there any way to have splunk show the HTML formatted log data in search results? I'm planning on parsing out a few of the basics like error code, location, user etc as standard splunk variables, but the detail will need to remain in HTML.

If HTML can't be used, how do other people deal with large amounts of debug data? Can splunk better process XML for instance?

Tags (2)
1 Solution

Michael_Wilde
Splunk Employee
Splunk Employee

Any reason why your output has to be HTML? I have used a command line browser, such as lynx or elinks wired up to a scripted input in Splunk that polls a webpage and retrns the results...something like . I actually didn't want all the HTML tags in my data. Why do you need them in yours?

If you look at http server status, in my case I wanted that big table of http objects and all the stats. With lynx I rendered it, with awk, I formatted each line with a timestamp and key value pairs. Might be an approach for you if you have control over how the log comes out.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

As Michael said, it would be better if you didn't have it in HTML.

Nevertheless, Splunk can take and index the HTML just fine, and should be able to do field extractions on the data as well. (Your regexes will be ugly to get around the HTML.) The HTML will not be rendered in the Splunk UI, but I suppose that should be unnecessary, since what you do with Splunk is extract the data, then analyze and display it from there.

gkanapathy
Splunk Employee
Splunk Employee

That's probably better. Splunk is really for collecting data out of text files, then analyzing and processing the data, and the UI is made for that, not for rendering documents. In principle, a whole new UI could be built to do whatever with the indexed data, but that's not really the expected use of the standard web UI.

0 Karma

JShood
Explorer

Thanks for the suggestions. I think I may be trying to use splunk for something it's not intended. For each error in our web app we do a detailed dump of all variables, stack traces and sql states at the time of the error. This is rendered in a 500k+ HTML page. My intention was to add some header info that could be easily parsed by splunk and then from the splunk UI be able to view the nicely formatted HTML dump created by our application. I'm now considering splitting the log into two files, an overview for splunk to process and then a separate detailed dump. Any other suggestions?

0 Karma

Michael_Wilde
Splunk Employee
Splunk Employee

Any reason why your output has to be HTML? I have used a command line browser, such as lynx or elinks wired up to a scripted input in Splunk that polls a webpage and retrns the results...something like . I actually didn't want all the HTML tags in my data. Why do you need them in yours?

If you look at http server status, in my case I wanted that big table of http objects and all the stats. With lynx I rendered it, with awk, I formatted each line with a timestamp and key value pairs. Might be an approach for you if you have control over how the log comes out.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...