Splunk Search

Obtaining XML from URL

helius
Path Finder

Search Head: V6.2
Goal: Obtain XML data from URL, which is dynamically created with IDs set in search string.

Search string: index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id

I want to then create and, from what I've read, use urldecode & xmlkv :

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv

I should then be able to rex this and get some sort of result into file_size field:

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv | rex field=_raw "(?<file_size>\d+)"

This produces the field file_size, but it's "2" for everything, regardless of the actual number/file_size in the xml doc.

I don't think I'm doing something right... Can someone point me into obtaining the value in the URL's XML?

Here is an example of the XML:

<general>
<duration>30mn 4s</duration>
<format>MPEG-TS</format>
<id>2 (0x2)</id>
<overall_bit_rate>3 750 Kbps</overall_bit_rate>
<file_size>807 MiB</file_size>
<overall_bit_rate_mode>Constant</overall_bit_rate_mode>
</general>

Thanks in advance!

0 Karma
1 Solution

helius
Path Finder

It turns out you cannot do what I'm trying to do in Splunk. Sure would be nice, but I'll just go ahead and index these remote XML files and use xpath to get what I want.

View solution in original post

0 Karma

helius
Path Finder

It turns out you cannot do what I'm trying to do in Splunk. Sure would be nice, but I'll just go ahead and index these remote XML files and use xpath to get what I want.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Your regex string is taking the first number it finds and putting it into the file_size field, which is probably not the desired result. Try this instead:

... | rex "(?<file_size><file_size>(\d+).+</file_size>)"

If you want more than just the numbers, use this:

... | rex "(?<file_size><file_size>(.+)</file_size>)"
---
If this reply helps you, Karma would be appreciated.
0 Karma

helius
Path Finder

Thanks Rich, I tried that and just receive the http://kemgr-ch2-a1p.domain.com/jobs/81283.xml URL back, no file_size number as a field.

Here was my search string:

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv | rex "(?(\d+).+)"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The urldecode() function does not send an http query, it merely makes a URL easier to read. That means the xmlkv statement is parsing your URL, not an XML string. That, in turn, means rex will not find 'file_size' - in fact, the current regex string will return the '2' from your URL.

---
If this reply helps you, Karma would be appreciated.

helius
Path Finder

Aaahhh, that makes much more sense now. Is something like what I'm trying to do possible without indexing these XML files?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think you'll have to index the files.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...