timestamp not being recognized for Proofpoint Add-On

[jeff]

Timestamps are not properly being parsed using v1.0.0 of the TA-pps_ondemand app.

01-16-2019 02:56:09.819 -0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Wed Jan 16 02:28:43 2019). Context: source=proofpoint_message_log|host=sp-01|pps_messagelog|

[jeff · Accepted solution]

Add the following to local/props.conf

[pps_messagelog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

[pps_maillog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

added 1 May 2019

Add the following to local/limits.conf

[spath]
extract_all=true
extraction_cutoff=30000

note: version 1.0.2 of the app added "DATETIME_CONFIG = CURRENT" to default/props.conf, which overrides these settings. I edited default to remove that configuration, but "proper" method would be to add "DATETIME_CONFIG = /etc/datetime.xml" to local/props.conf to restore the default. I have reported this to the developers and they responded they're workig on a revised version to include proper date/time extraction.

[ashutoshab]

I also had the same issue. I tried setting up timestamp configuration myself but i messed something. Later found that there was slight mistake in the Regex for TIME_PREFIX. I used your suggestion and now I am perfect.

Thanks.

Upvoted.