All Apps and Add-ons

Microsoft Office 365 Reporting Add-on for Splunk: Why are the message time and Splunk query time different?

davidblizzard
Explorer

Good morning all.

I am getting message trace data using this Add-on, however the times are off. The result time shows the time that the data was received, not the actual date/time the message was received. Can anyone help me get them matched up? This is the only app with this issue.

Thanks.

jconger
Splunk Employee
Splunk Employee

You may need top copy the props.conf from the add-on to your search head(s). Here are the relevant time-related settings in props.conf from the add-on:

[ms:o365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z
TIME_PREFIX = "Received": "

Does the TIME_FORMAT match what you are seeing in your data?

davidblizzard
Explorer

Thanks for the response. This is a single Windows server installation, so all components are on the same server. Here are my props file entries you are referring to:

[ms:o365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = 0
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
TIME_PREFIX = "DateReceived":

I changed the lookahead to 19 while troubleshooting based on similar issues reported (it was initially 30). I would have no issue setting that back.

The only other difference i see here is in the time_format, you show a % between S and Z. Mine is not separated by a %.

0 Karma

jconger
Splunk Employee
Splunk Employee

The TIME_PREFIX should be "Received": "
The MAX_TIMESTAMP_LOOKAHEAD is too short also. The timestamps in the data can be 27 characters long.

0 Karma

davidblizzard
Explorer

Thank you. I have upgraded to the latest version and configured everything appropriately. Things are going well. The only issue I'm having now is the timezone. The query time matches the UTC time in the trace. I'd like it to display Central timezone. I've tried adding TZ = US/CENTRAL as well as TZ = America/Chicago to the props file, still not working.

0 Karma

medunmeyer
Explorer

Try using TZ=UTC. That matches the TZ of the O365MessageTrace logs. Then they will display at US/Central if your servers/clients are set for that TimeZone.

0 Karma

davidblizzard
Explorer

Good morning medunmeyer. That is not working either. More info below. Any help would be appreciated. Thanks.

My props file is below. I've tried setting TZ = UTC, America/Chicago, US/Central to no avail.
I have also removed the %Z from the Time_Format line thinking it may essentially disable the TZ statement.

The 'Received' data from the trace looks like this:

Received: 2018-10-24T09:33:56.8119313

Splunk Query time looks like this:

10/24/18
9:33:56.000 AM

Should be:

10/24/18
4:33:56.000 AM

[ms:o365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = 0
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z
TZ = UTC
TIME_PREFIX = "Received": "
category = Splunk App Add-on Builder
pulldown_type = 1
EVAL-vendor_product = Microsoft Office 365
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_1 = RecipientAddress AS recipient
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_10 = SenderAddress AS orig_src
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_11 = SenderAddress AS src_user
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_12 = ToIP AS dest
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_13 = FromIP AS src
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_4 = MessageId AS message_id
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_5 = Subject AS subject
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_6 = Size AS size
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_7 = MessageTraceId AS internal_message_id
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_8 = Status AS action
FIELDALIAS-aob_gen_ms_o365_reporting_messagetrace_alias_9 = RecipientAddress AS orig_recipient

[ms:o365:reporting:mailtraffic]
SHOULD_LINEMERGE = 0
category = Splunk App Add-on Builder
pulldown_type = 1

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...