Splunk truncating XML

[stephenho]

Hi All,
I'm having a play with webservices at the moment and I'm using a python script to retrieve weather information from a an online site. My python script outputs the following at the moment : -

<?xml version="1.0" encoding="utf-16"?>
<CurrentWeather>
  <Location>Dubbo, Australia (YSDU) 32-12S 148-34E 275M</Location>
  <Time>Oct 27, 2012 - 01:00 AM EDT / 2012.10.27 0500 UTC</Time>
  <Wind> from the SW (230 degrees) at 13 MPH (11 KT):0</Wind>
  <Visibility> greater than 7 mile(s):0</Visibility>
  <Temperature> 75 F (24 C)</Temperature>
  <DewPoint> 24 F (-4 C)</DewPoint>
  <RelativeHumidity> 15%</RelativeHumidity>
  <Pressure> 29.97 in. Hg (1015 hPa)</Pressure>
  <Status>Success</Status>
</CurrentWeather>

However, when I run it in splunk, my location and time tags are truncated to the comma. So i'll only see

 <?xml version="1.0" encoding="utf-16"?>
    <CurrentWeather>
      <Location>Dubbo
      <Time>Oct 27
      <Wind> from the SW (230 degrees) at 13 MPH (11 KT):0</Wind>
      <Visibility> greater than 7 mile(s):0</Visibility>
      <Temperature> 75 F (24 C)</Temperature>
      <DewPoint> 24 F (-4 C)</DewPoint>
      <RelativeHumidity> 15%</RelativeHumidity>
      <Pressure> 29.97 in. Hg (1015 hPa)</Pressure>
      <Status>Success</Status>
    </CurrentWeather>

xmlkv also doesn't seem to be finding any of the other tags, but can I assume it's due to the errors from the first 2 tags?

Cheers,

Steve

[gkanapathy]

I don't really understand how you "run it in Splunk". It would be helpful if you explained what you're doing and exactly where you're seeing this.

Also, is your Python script really outputting in utf-16 encoding?

Also, spath is preferred over xmlkv.

[gkanapathy]

Okay, custom search commands and scripts don't work this way. You are expected to output in CSV format here, with the raw data in the field _raw. This is not like with scripted inputs, where you would simply write the raw to stdout (as you did here). You should probably look at the docs http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/WriteaPythonsearchcommand as well as the shipped scripts in $SPLUNK_HOME/etc/apps/search/bin

[stephenho]

import sys

if len(sys.argv) != 3:
    print "Usage: sysarglimi.py [city] [country]"
    sys.exit(0)

cityf = sys.argv[1]
countryf = sys.argv[2]

from suds.client import Client
url = "http://www.webservicex.net/globalweather.asmx?WSDL"
client = Client(url)
result2 = client.service.GetWeather(cityf,countryf)
print result2

[stephenho]

Hi Gkanapathy,

I wrote a python script, and put an entry into the commands.conf file.

And when I run that script in command line, I get the first output as mentioned in my original post.

However, when I run it in splunk via

|script python findWeather dubbo Australia

I get the truncated version of the of the XML on the first 2 tags. Due to line restrictions, I'll paste the code of my python script in the next comment.

Thanks,

Steve