topic Re: ms:o365:reporting:messagetrace props.conf settings for timestamp recognition in Splunk Enterprise Security

ms:o365:reporting:messagetrace props.conf settings for timestamp recognition

jacodutoit — Tue, 29 Sep 2020 19:28:38 GMT

Hi Splunkers

Does anyone know the correct settings for the props.conf file of the TA-MS_O365_Reporting add-on that ensures that the "Time" field is extracted and displayed in my time zone (Pacific/Auckland)? It currently displays the extracted "Time" field in UTC.

Using the default settings of the props.conf file as below doesn't convert the extracted field to my timezone:
[microsoft:office365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = 0
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
TIME_PREFIX = "DateReceived": "

I've also worked through the answers discussed below without success:
https://answers.splunk.com/answers/626095/new-time-format-has-z-on-the-end-did-you-mean-z-fo.html

Additionally, I extracted the search results to a csv file and used the "Add Data" interface on my search head and heavy weight forwarder (where the add-on is configured) to add the data to verify the extracted fields. When I select the source type as "ms:o365:reporting:messagetrace" it does convert the UTC time to my timezone in the "Time" field. However, during a search it does not and uses the UTC time as the "Time".

Below is what my current props.conf file looks like:
[ms:o365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = 0
TIME_FORMAT = %Y-%m-%dT%H:%M:%S %Z
TIME_PREFIX = "DateReceived":
category = Splunk App Add-on Builder
pulldown_type = 1

I've also tried the following TIME_FORMAT options:
1. %Y-%m-%dT%H:%M:%S%Z
2. %Y-%m-%dT%H:%M:%S

I've also tried the following TIME_PREFIX options:
1. "DateReceived": "
2. ""DateReceived"": ""

I've changed the MAX_TIMESTAMP_LOOKAHEAD to 100.

I've added "TZ = UTC" and tried "TZ = Pacific/Auckland".

I also used other Splunk accounts and verified the time zone settings in the user account options.

Any assistance in this regards will be highly appreciated.

Regards

Re: ms:o365:reporting:messagetrace props.conf settings for timestamp recognition

fredzhang — Wed, 30 Sep 2020 03:33:50 GMT

tested in my case and it works (version 1.10) . hope it work for you:

TA-MS_O365_Reporting/local/props.conf:
[ms:o365:reporting:messagetrace]
TZ = UTC

Re: ms:o365:reporting:messagetrace props.conf settings for timestamp recognition

victormelnikftb — Wed, 30 Sep 2020 03:57:36 GMT

The solution below worked for our organization:

In the props.conf file located in [SPLUNK_HOME]\etc\apps\TA-MS_O365_Reporting\default, comment out the following:

[ms:o365:reporting:messagetrace]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = 0
*#TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z

TIME_PREFIX = "Received": "*

category = Splunk App Add-on Builder
pulldown_type = 1

Create a new props.conf in [SPLUNK_HOME]\etc\apps\TA-MS_O365_Reporting\local and add the following:

[ms:o365:reporting:messagetrace]
TZ = UTC
MAX_TIMESTAMP_LOOKAHEAD = 300
TIME_PREFIX = "Received":\s*"

Thanks! Hope this solution helps.