Splunk support for AWX Ansible: Integrating Splunk with AWX for Job Monitoring

[Narendra_Rao]

I'm currently working on integrating Splunk with AWX to monitor Ansible automation jobs. I'm looking for guidance on the best practices for sending AWX job logs to Splunk. Specifically, I'm interested in:

• Any existing plugins or recommended methods for forwarding AWX logs to Splunk.
• How to differentiate logs from QA and production environments within Splunk.
• Examples of SPL queries to identify failed jobs or performance metrics.

Any advice or resources you could share would be greatly appreciated.
Thanks.

[tscroggins]

Hi @Narendra_Rao,

AWX appears to support streaming logs directly to Splunk HTTP Event Collector. See https://ansible.readthedocs.io/projects/awx/en/latest/administration/logging.html#splunk and https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector or https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/UsetheHTTPEventCollector.

Differentiation by environment depends on your deployment architecture. If the host field isn't sufficient, the AWX log cluster_host_id field may be. You can define a simple lookup in Splunk to set, for example, an environment field based on the host or cluster_host_id field value.

I've not used AWX, but see https://ansible.readthedocs.io/projects/awx/en/latest/administration/logging.html for the AWX log schema. Job events and possible job status changes seem like a good starting point.

The app dashboards provide a few search examples that may be useful for building your own searches.