Note: This post outlines a proposed architecture and serves as an interest check. If we secure commitments from 100 interested members, the full build playbook and repository will go live. Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a fundamental misunderstanding: treating Zero Trust as a commodity you can buy. When subject matter experts (SMEs) view ZT strictly through the lens of their specific functional roles, the reflex is almost always to procure a commercial tool to check off a requirement. However, Zero Trust is not a single software suite. A true ZT solution is frequently a combination of technical, logical, or administrative controls designed to achieve specific operational outcomes rather than a vendor feature set. To successfully execute Zero Trust, organizations must evaluate every task within the context of its execution phase, target maturity level, and operational boundary. Translating frameworks like the DoD Zero Trust Execution Roadmap Courses of Action (COAs) requires looking beyond generic tool capabilities. Teams must break down high-level objectives into granular outcomes by analyzing how user behaviors, device postures, and network boundaries interact over time. Without this phase-aware perspective, organizations risk deploying expensive security software that fails to meet actual compliance and verification mandates. Enterprise Compliance Hub: Splunk App for Zero Trust Execution For Splunk customers seeking an enterprise-wide compliance hub, our open-source Splunk App for Zero Trust Execution directly bridges the gap between raw security data and audit readiness. By transforming DoD COA execution roadmaps into actionable lookup tables, ingested via the Splunk Lookup Editor from custom Excel files, the app structures its top navigation around the core ZT Pillars (User, Device, Application/Workload, Data, Network, Automation, Visibility). Individual tasks form the interactive tabs within each pillar. As SMEs populate updates and attach log-based evidence directly within the app, it automatically compiles an authoritative Proof of Delivery (PoD) ready for ISSO/CISO signature. If gaps are identified, the app dynamically generates ZT Milestones, converting Splunk into a centralized Integrated Master Schedule (IMS) tracker. Organizations that do not currently operate a SIEM capable of hosting custom applications can achieve the exact same structured tracking through Microsoft Planner. By leveraging JSON-formatted outcome definitions for each ZT task, teams can manually track task predecessors, successors, and stage progressions, such as Discovery, Phase 1, and Phase 2, using native Planner labels. The fundamental trade-off comes down to operational integration: Splunk provides an automated, telemetry-driven platform combining IMS tracking, live evidence ingestion, and automated PoD generation in a single interface, whereas Microsoft Planner serves as a lightweight, non-integrated tracking alternative for manual oversight. Real-Time Architecture Visualization: AWS-DFD-Visualizer (v2.8.3) To move from manual compliance tracking to real-time visual proof, our companion open-source project published on Splunkbase, AWS-DFD-Visualizer (v2.8.3), renders complex multi-cloud architecture data into dynamic, audit-ready Zero Trust topology blueprints. Built for NIST 800-207 and DoD Impact Level 5 (IL5) environments, AWS-DFD-Visualizer automatically categorizes live inventory into discrete operational tiers, including web servers, mid-tier applications, data stores, and containers. It then projects these assets across NIST 800-207 Three-Plane Swimlanes (Identity, Policy & Control, and Infrastructure/Data) to visually expose the core "P*" components of your architecture: Core "P*" Zero Trust Architectural Components PEP (Policy Enforcement Point) & PIP (Policy Information Point): Real-time posture checks fed by tools like ForeScout C2C, Trellix, Microsoft Defender for Endpoint (MDE), and Tenable. PDP (Policy Decision Point), PE (Policy Engine), PA (Policy Administrator) & PAP (Policy Administration Point): Orchestrated control logic anchored by SOAR playbooks and enterprise SIEM policies (Splunk/Sentinel). By auto-detecting CSP environments like AWS, Azure, or GCP and highlighting non-compliant paths, such as direct SSH/22 ingress, AWS-DFD-Visualizer provides immediate visual proof of policy enforcement across every workload tier. Community-Driven Build Strategy & Demand Validation To ensure maximum alignment with community needs, full feature rollout of the Splunk App for Zero Trust Execution follows a demand-validated engineering model: Validating Market Demand: Engagement with this community blog post alongside download metrics from our published AWS-DFD-Visualizer app act as direct demand indicators for expanding the Zero Trust Execution platform. Telemetry-Informed Roadmap: Download velocity and active installation metrics directly shape our development priorities, ensuring we build features based on real-world adoption patterns. Continuous Feedback Loops: Community feedback helps us continuously refine whether security teams prefer automated analytics in Splunk or lightweight manual workflows like Microsoft Planner. Future-Proofing Compliance with TKU Releases To support dynamic multi-cloud deployments, our release cycle incorporates regular Technology Knowledge Updates (TKUs). These updates maintain the integrity of live inventory tags, update non-compliant port heuristics (such as SSH/22 ingress rules), and expand stencil libraries across AWS, Azure, and GCP. By pairing TKU maintenance with telemetry-driven feedback from Splunkbase, we ensure your Zero Trust baseline remains aligned with evolving NIST 800-207 standards and DoD Impact Level 5 (IL5) requirements. Conclusion & Operational Impact Ultimately, Zero Trust is an operational evidence discipline, not a procurement exercise. Purchasing tools like SIEMs, EDRs, or C2C platforms is only the first step; the true measure of Zero Trust lies in your ability to continuously document, map, and prove that your administrative and technical controls satisfy each task's required outcome. Whether you choose the fully integrated, automated analytics model inside Splunk or a lightweight manual workflow in Microsoft Planner, establishing a central repository for proof of delivery ensures your security posture remains defensible, compliant, and resilient against evolving threats. ... View more

Asset and Identities Workflow.png Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are misattributed, investigations stall, and compliance reporting becomes unreliable. Yet practitioners face recurring challenges: inconsistent data across sources, missing attributes, schema drift, and conflicts between authoritative systems. This blog provides a practical guide for engineers and analysts building and maintaining asset and identity frameworks in Splunk. It walks through common issues and their solutions, demonstrates how to leverage KV lookups for normalization, and offers a troubleshooting playbook to ensure frameworks remain deterministic, auditable, and contributor friendly. Key Topics Covered:  Traditional and cloud-native infrastructure identity challenges KV lookup architecture and performance optimization Comprehensive troubleshooting playbook with SPL examples CMDB integration and identity lifecycle management Metrics, validation, and governance frameworks Introduction In modern enterprises, assets and identities are described by multiple systems:  Endpoint protection platforms (e.g., CrowdStrike, McAfee): Provide MAC addresses and hostnames. Network infrastructure (e.g., DHCP, firewalls): Provide IP addresses but lack persistent identifiers. Identity providers (e.g., Active Directory, HR systems): Provide usernames, employee IDs, or email aliases. Vulnerability scanners (e.g., Qualys, Tenable): Add asset tags and risk scores. Cloud platforms (AWS, Azure, GCP): Provide instance IDs, tags, and IAM roles. Container orchestration (Kubernetes, Docker): Generate ephemeral identities. CMDBs (ServiceNow, Jira): Serve as authoritative asset inventory sources. The challenge is not the lack of data, but the fragmentation of attributes. Splunk's KV store lookups provide a powerful mechanism to unify these attributes into a single, authoritative mapping. Common Challenges and Solutions Traditional Infrastructure Challenges Issue Impact Solution Source A lacks MAC, Source B lacks hostname Incomplete correlation  KV lookup merges attributes (MAC ↔ hostname) Duplicate identities across HR/IAM/VPN Conflicting user resolution  Normalize usernames; canonical identity KV keyed on employee ID  Dynamic IPs on laptops/mobiles Alerts tied to stale IPs  Map IP → MAC → hostname via DHCP; refresh KV lookup hourly Schema drift (user_id vs uid vs userid) Breaks correlation searches  Field normalization macros + schema mapping lookup Multi-homed servers (multiple NICs) Incomplete asset picture  Store all IPs as multi-value field with primary designation VDI non-persistent desktops Rotating hostnames break tracking Correlate via Citrix/Horizon session ID → username Building Asset and Identity KV Lookups Principles Canonical Keys: Choose stable identifiers (MAC for assets, Employee ID for identities). Multi-Source Enrichment: Merge attributes from multiple sources into a single KV record. Scheduled Updates: Refresh KV lookups based on data volatility. Auditability: Track source-of-truth and last_updated timestamps. Graceful Degradation: Use fallback identifiers when primary is unavailable. Asset and Identities Workflow 1.png Understanding the "Field Limits" Error When Splunk Enterprise Security (ES) merges data from disparate sources, it populates the asset_lookup_by_str, asset_lookup_by_cidr, and identity_lookup_expanded searches. If a single identity accumulates more attributes than the system's defined limit, the Identity - Asset Truncations search triggers a warning. The Problem: Alerts show null values or stale hostnames because critical data is dropped during the merge. The Insight: This is typically caused by a mismatch between data volume and the Multivalued field restriction in settings. Step 1: The Diagnostic SPL Identify which field is exceeding the limit with this query: Key Action: Note the field with the highest max_values. If it exceeds your system limit (default is often 10), it is the culprit. | inputlookup asset_master.csv | foreach * [ eval <<FIELD>>_count = if(isnotnull('<<FIELD>>'), mvcount('<<FIELD>>'), 0) ] | stats max(*_count) as * | transpose 0 column_name="field" header_field=column | rename "row 1" as max_values | where max_values > 0 | sort - max_values Step 2: Resolution (UI-Based Fix) Navigate to Configure > Data Enrichment > Asset and Identity Management. Select the Asset Fields or Identity Fields tab. Locate the field identified in Step 1. Change the Multivalued setting to match your SPL results (e.g., increase to 17) Re-run the Identity - Asset and Identity Correlation search. Step 3: Advanced Remediation & Governance Data Cleaning: Filter vulnerability feeds to only include "Critical" or "High" issues Source Precedence: Define clear rules (e.g., CMDB > AD > DHCP) Scheduled Pruning: Use mvfilter or mvindex to keep only the most recent 30 days of history SOC Analyst Runbook Additions Weekly: Review dashboards for resolution errors and check identity_manager.log. Monthly: Audit KPIs including Coverage Rate and Freshness. Verification: Run | inputlookup identities.csv to ensure data is populating correctly ... View more