All Apps and Add-ons

Why are there Cisco ESA Field Extraction Errors and CIM Naming Issues in Splunk Add-on for Cisco ESA?

Communicator

The Splunk Add-on for Cisco ESA doesn't extract multiple fields correctly. I discovered a couple, but there are likely other issues in the field extractions and naming.

  • CIM "subject" is named as "message_subject"
  • Source and destination IPs are mixed up for incoming connections
  • For an ESA email gateway, should the IP address of the gateway IP be used as both the "src_ip" and "dest_ip" for a delivered message?
  • Logs with multiple "internal_message_id" values only extract one value

https://splunkbase.splunk.com/app/1761/

Communicator

This original question is a good example on how the transition to the new backend degraded previous posts.

This is what it should look like, with bold/green to highlight the original data that was incorrectly modified during the transition to Khoros:

The Splunk Add-on for Cisco ESA doesn't extract multiple fields correctly. I discovered a couple, but there are likely other issues in the field extractions and naming.

  • CIM "subject" is named as "message_subject"
  • Source and destination IPs are mixed up for incoming connections
  • For an ESA email gateway, should the IP address of the gateway IP be used as both the "src_ip" and "dest_ip" for a delivered message?
  • Logs with multiple "internal_message_id" values only extract one value

https://splunkbase.splunk.com/app/1761/

The app was updated a couple months after the initial question.

- Subject is aliased to CIM
- DCID destinations IPs are mapped correctly
- For delivered messages, the delivery IP is labeled as del_ip_address  (Khoros doesn't allow new inline code blocks, but previous posts retain them)
- The internal_message_id is extracted for all related logs

There are still other issues regarding SPF, DKIM, and DMARC parsing, and subject and filename decoding, and issues when Cisco truncates fields to 1024 bytes (the syslog output is then the syslog header length, plus the 1024 bytes of data).

I believe that Cisco will soon include an option to output JSON logs from the ESA appliances, which may alleviate a many of these ESA log parsing issues.

I recommend looking at Jorrit Folmer's app that wraps the messages into a summary. It summarizes almost all of the log lines for each messages into a single line that can be searched against using multiple criteria, without the heavy invocation of transaction that the Cisco ESA app defaults to.

https://github.com/jorritfolmer/TA-messagetracking-for-cisco-esa

Tags (2)
0 Karma

Champion

Can you tell how you install this add-on? Could you please send configuration files used? Dod you used custom inputs?

0 Karma

Communicator
  1. The state of installation of the add-on does not address this issue, as the configuration's extractions are working with the issues noted above.
  2. The configuration files are readily available from the link initially provided, see its transforms.conf.
  3. Custom extractions were created to fix the issue locally.
0 Karma

New Member

Hello,

Can you provide me the custom extractions you created to fix this issue. I\'m facing the same issue now.

Thanks in advance.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!