AI-Powered Observability and Observability for AI

Community Office Hours

AI-Powered Observability and Observability for AI

1 Comment
Cover Images - Office Hours (10) copy.jpg
Published on ‎02-13-2026 09:44 AM by Community Manager | Updated on ‎03-25-2026 07:19 AM

[Watch On-Demand] This thread for the Community Office Hours session on AI-Powered Observability and Observability for AI on March 19th 2026 | 11:00am - 12:00pm PT

Ask the experts at Community Office Hours! An ongoing series where technical Splunk experts answer questions and provide how-to guidance on various Splunk product and use case topics.

What can I ask in this AMA?

  • How can I leverage Splunk’s AI capabilities to proactively detect anomalies and remediate incidents?
  • What are some common challenges or limitations when implementing AI for observability at scale?
  • How can I track and optimize cost and resource usage across my AI apps and services?

Please submit your questions at registration.
You can also head to the #office-hours Community Slack channel to ask questions (sign-in with SSO here).

Pre-submitted questions will be prioritized.
After that, we will open the floor up to live Q&A with meeting participants.

We look forward to connecting with you!



0 Karma
ArifV
Splunk Employee

Hi everyone! Here are a few questions from the session, as well as the link to the on-demand recording (get the full Q&A deck and recording in the #office-hours Slack channel as well) 

Q1: What does the transition path look like from reactive alerting to closed-loop, resilience-oriented feedback systems in AI-driven environments?

A1: We are evolving from reactive OOTB and custom detector alerts to a human-in-the-loop model where AI agents use metrics, traces, and logs for automated troubleshooting. 

By applying security guardrails and granular permissions, agents learn safe remediation actions over time. This transition establishes the foundation for closed-loop, resilience-oriented feedback systems

Q2: When AI agents initiate autonomous remediation, where are institutions anchoring the accountability surface - configuration, model approval, or runtime intervention?

A2: When AI agents initiate autonomous remediation, institutions are increasingly anchoring the accountability surface in runtime intervention.

While configuration and model approval provide the necessary foundation for safety, the most critical accountability anchor occurs when the troubleshooting agent presents its findings and proposed resolution to a human operator. By requiring human authentication before the remediation agent executes any changes, the institution ensures that the final decision-making authority remains with a human, effectively bridging the gap between autonomous execution and operational oversight.

Q3: How does the MCP server differ from the Splunk Observability AI Assistant I’m already using?

A3:The Splunk Observability's AI Assistant is a built-in chat interface designed for human users to interact with Splunk Observability data, accelerating troubleshooting directly in the Splunk Observability Cloud UI using natural language.

In contrast, the MCP server is a backend protocol that allows external AI agents (like Claude or custom agents) to programmatically access Splunk data and tools, enabling automated, cross-platform workflows that operate independently of the Splunk UI.

Documentation:

Q4: It would be nice to see a demo on using Splunk to find shadow IT/OT and AI?

A4:As part of Cisco AI Defense, AI Cloud Visibility automatically identifies AI assets within custom-built applications across your distributed environment, including unsanctioned workloads. This provides a centralized view, enabling you to inventory and assess your organization's AI security exposure.

Coming soon, AI Agent Monitoring will be integrated with Cisco AI Defense to pinpoint AI risks and vulnerabilities.

Documentation: