All Apps and Add-ons

Why is MS Cloud Services app not collecting recent data from Storage Table API?


I have come across a unique issue with the MS Cloud Services app when assigning an input for a single storage table. The API works as expected when I initially set the input with a Start Time. However, the data does not continue to collect/ingest beyond the timestamp when the input is configured, unless there is manually intervention/manipulation on my part to the input Start Time.

Example: If I set a Start Time to 2 weeks prior and save the input, it will collect data from the Storage Table at 2 weeks prior up to the time the input is saved in Splunk. The input will generate 0 results after that time.

I checked in Azure Storage Explorer and the Table in question continues to write new entries in the same format. I have confirmed Splunk can see that data because it will start collecting new data only after I manually update the Start Time in the input.

I checked the mscs:storage:table:log and there are no errors with the API input functionally and it shows attempts at the designated interval(5 minutes).

Historically with this input, I've had success by leaving the Start Time to the default(30 days) and setting the table list to * to collect everything. However, this table is part of a very large blob that cannot be pulled in the same fashion.

I'm hoping to get some ideas about what could be causing this break in log collection and see if there is something I may be overlooking. Any input would be greatly appreciated.

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...