I am trying to use the history feature to retrieve daily weather values for San Francisco & Dallas using the search query :
sourcetype="wunderground" source="wunderground:SF"
sourcetype="wunderground" source="wunderground:Dallas"
and my json configuration file is
API feature = history
{ "country": "CA", "city": "San Francisco", "from":"2007-01-01", "to":"2015-05-01"}
{ "city": "Dallas", "country": "TX" ,"from":"2014-12-01", "to":"2014-12-10"}
The results for this vary in number of events from as low as 70 to sometimes 200-300 but they show results only for the year 2007 that also not the complete year.
The default time that Splunk looks back is 2000 Days. Splunk consumes the API data and sets the _time field to the time of the Wunderground collection time. So therefore, we need to increase the time.
In local/props.conf, add this:
[wunderground]
MAX_DAYS_AGO = 10000
And restart. The data should start appearing in the correct time buckets.
So I ran my search again using
sourcetype="wunderground" source="wunderground:SF"
for the config file - { "country": "CA", "city": "San Francisco", "from":"2007-01-01", "to":"2015-05-01"}
First I got 25 events for Jan 2007 in the results, then after running it again after a brief period I got more values for Jan & feb around 800. I repeated this process of running after a few intervals and got around 800 events per month for uptil May 2007.
Then when I ran the search again, it returned more values for January & February (the counts went up from 800 to around 2000) but they were all duplicates. Shortly after I received an email from wunderground stating I had exceeded the number of daily calls (500 since I am a free user). Also for 1st June 2015 I get events returned but they do not contain any data. What am I doing incorrectly? Please check the screenshots .
drive.google.com/file/d/0B8IDZa4NAwfqXzhUNmVydlFOQWs/view?usp=sharing
drive.google.com/file/d/0B8IDZa4NAwfqMjR2YkhUc1lFLUE/view?usp=sharing
Thanks for the reply! So I tried that and the results that followed weren't different than what I was getting earlier. Right now I got values only for the year 2007. I also tried this for the time range 2014-12-01 to 2014-12-22 as pointed out by another answer in this group. But I still didn't get all the values.