Dashboards & Visualizations

How can I apply XML DMARC report metadata values to each event in the report?

Engager

I'm ingesting XML DMARC reports into Splunk, but the individual events aren't very useful without including things like begin_date, end_date, org_name, email and report_id in each event. Those values only exist in a "metadata" section at the top of the report. How can I take those values that only occur once in the report and include them in each event?

Here's a sample of the XML data I'm ingesting:

<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
  <report_metadata>
    <org_name>emailsrvr.com</org_name>
    <email>dmarc_reports@emailsrvr.com</email>
    <extra_contact_info>http://emailsrvr.com</extra_contact_info>
    <report_id>1a292ea2-d440-4985-a969-839778bceac1</report_id>
    <date_range>
      <begin>1487289600</begin>
      <end>1487376000</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>mycompany.com</domain>
    <adkim>r</adkim>
    <aspf>r</aspf>
    <p>none</p>
    <sp>none</sp>
    <pct>100</pct>
  </policy_published>
  <record>
   <row>
     <source_ip>192.168.x.x</source_ip>
     <count>1</count>
     <policy_evaluated>
       <disposition>none</disposition>
       <dkim>fail</dkim>
       <spf>pass</spf>
     </policy_evaluated>
   </row>
   <identifiers>
     <header_from>mycompany.com</header_from>
   </identifiers>
   <auth_results>
     <spf>
       <domain>mycompany.com</domain>
       <result>pass</result>
     </spf>
   </auth_results>
  </record>
  <record>
   <row>
     <source_ip>192.168.x.x</source_ip>
     <count>1</count>
     <policy_evaluated>
       <disposition>none</disposition>
       <dkim>pass</dkim>
       <spf>fail</spf>
     </policy_evaluated>
   </row>
   <identifiers>
     <header_from>mycompany.com</header_from>
   </identifiers>
   <auth_results>
     <spf>
       <domain>mycompany.com</domain>
       <result>temperror</result>
     </spf>
   </auth_results>
  </record>

I haven't been able to figure out how to pull the date and org_name fields out of the report_metadata section and put them into into each individual event. So ideally, I'd like my events to look something like:

  report_id=1a292ea2-d440-4985-a969-839778bceac1, date_begin=1487289600, date_end=1487376000, org_name=emailsrvr.com, source_ip=192.168.x.x, disposition=none, dkim=fail, spf=pass, header_from=mycompany.com
  report_id=1a292ea2-d440-4985-a969-839778bceac1, date_begin=1487289600, date_end=1487376000, org_name=emailsrvr.com, source_ip=192.168.x.x, disposition=none, dkim=pass, spf=fail, header_from=mycompany.com

Some questions:
1. Should I ingest the whole XML file as a single event and then do the processing at search time, or should I use XML_KV and break events on the tag?
2. How do I parse out the values in the report_metadata field and apply them to each event?

Here's my current sourcetype configuration for this data source:

[dmarc_xml]
DATETIME_CONFIG =
LINE_BREAKER = (<record>)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
TIME_FORMAT = %s
TIME_PREFIX = <end>
category = Email
description = DMARC XML Reports
disabled = false
pulldown_type = true
BREAK_ONLY_BEFORE = (<record>)
KV_MODE = xml

Thanks in advance for your help!

0 Karma

Path Finder

Hi did you make any headway in this issue ? looking to do something similar with DMARC

0 Karma

Explorer

Hi,

Not a Splunk solution, but perhaps one of these scripts might help:

0 Karma