All Apps and Add-ons

TA for Wunderground: Why are all the values not returned using the history feature?

agoriawala_splu
Splunk Employee
Splunk Employee

I am trying to use the history feature to retrieve daily weather values for San Francisco & Dallas using the search query :

sourcetype="wunderground" source="wunderground:SF"

sourcetype="wunderground" source="wunderground:Dallas"

and my json configuration file is

API feature = history

{ "country": "CA", "city": "San Francisco", "from":"2007-01-01", "to":"2015-05-01"}

{ "city": "Dallas", "country": "TX" ,"from":"2014-12-01", "to":"2014-12-10"}

The results for this vary in number of events from as low as 70 to sometimes 200-300 but they show results only for the year 2007 that also not the complete year.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

The default time that Splunk looks back is 2000 Days. Splunk consumes the API data and sets the _time field to the time of the Wunderground collection time. So therefore, we need to increase the time.

In local/props.conf, add this:

[wunderground]
MAX_DAYS_AGO = 10000

And restart. The data should start appearing in the correct time buckets.

0 Karma

agoriawala_splu
Splunk Employee
Splunk Employee

So I ran my search again using
sourcetype="wunderground" source="wunderground:SF"
for the config file - { "country": "CA", "city": "San Francisco", "from":"2007-01-01", "to":"2015-05-01"}

First I got 25 events for Jan 2007 in the results, then after running it again after a brief period I got more values for Jan & feb around 800. I repeated this process of running after a few intervals and got around 800 events per month for uptil May 2007.

Then when I ran the search again, it returned more values for January & February (the counts went up from 800 to around 2000) but they were all duplicates. Shortly after I received an email from wunderground stating I had exceeded the number of daily calls (500 since I am a free user). Also for 1st June 2015 I get events returned but they do not contain any data. What am I doing incorrectly? Please check the screenshots alt text.alt text

drive.google.com/file/d/0B8IDZa4NAwfqXzhUNmVydlFOQWs/view?usp=sharing
drive.google.com/file/d/0B8IDZa4NAwfqMjR2YkhUc1lFLUE/view?usp=sharing

0 Karma

agoriawala_splu
Splunk Employee
Splunk Employee

Thanks for the reply! So I tried that and the results that followed weren't different than what I was getting earlier. Right now I got values only for the year 2007. I also tried this for the time range 2014-12-01 to 2014-12-22 as pointed out by another answer in this group. But I still didn't get all the values.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...