Splunk Search

Obtaining XML from URL

helius
Path Finder

Search Head: V6.2
Goal: Obtain XML data from URL, which is dynamically created with IDs set in search string.

Search string: index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id

I want to then create and, from what I've read, use urldecode & xmlkv :

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv

I should then be able to rex this and get some sort of result into file_size field:

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv | rex field=_raw "(?<file_size>\d+)"

This produces the field file_size, but it's "2" for everything, regardless of the actual number/file_size in the xml doc.

I don't think I'm doing something right... Can someone point me into obtaining the value in the URL's XML?

Here is an example of the XML:

<general>
<duration>30mn 4s</duration>
<format>MPEG-TS</format>
<id>2 (0x2)</id>
<overall_bit_rate>3 750 Kbps</overall_bit_rate>
<file_size>807 MiB</file_size>
<overall_bit_rate_mode>Constant</overall_bit_rate_mode>
</general>

Thanks in advance!

0 Karma
1 Solution

helius
Path Finder

It turns out you cannot do what I'm trying to do in Splunk. Sure would be nice, but I'll just go ahead and index these remote XML files and use xpath to get what I want.

View solution in original post

0 Karma

helius
Path Finder

It turns out you cannot do what I'm trying to do in Splunk. Sure would be nice, but I'll just go ahead and index these remote XML files and use xpath to get what I want.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Your regex string is taking the first number it finds and putting it into the file_size field, which is probably not the desired result. Try this instead:

... | rex "(?<file_size><file_size>(\d+).+</file_size>)"

If you want more than just the numbers, use this:

... | rex "(?<file_size><file_size>(.+)</file_size>)"
---
If this reply helps you, Karma would be appreciated.
0 Karma

helius
Path Finder

Thanks Rich, I tried that and just receive the http://kemgr-ch2-a1p.domain.com/jobs/81283.xml URL back, no file_size number as a field.

Here was my search string:

index=content_eng source="dbmon-tail://kemgr-a1p/Jobs General" | eval JobID=id | eval _raw=urldecode("http://kemgr-ch2-a1p.domain.com/jobs/"+JobID+".xml") | xmlkv | rex "(?(\d+).+)"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The urldecode() function does not send an http query, it merely makes a URL easier to read. That means the xmlkv statement is parsing your URL, not an XML string. That, in turn, means rex will not find 'file_size' - in fact, the current regex string will return the '2' from your URL.

---
If this reply helps you, Karma would be appreciated.

helius
Path Finder

Aaahhh, that makes much more sense now. Is something like what I'm trying to do possible without indexing these XML files?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think you'll have to index the files.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...