All Apps and Add-ons

timestamp not being recognized for Proofpoint Add-On

jeff
Contributor

Timestamps are not properly being parsed using v1.0.0 of the TA-pps_ondemand app.

01-16-2019 02:56:09.819 -0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Wed Jan 16 02:28:43 2019). Context: source=proofpoint_message_log|host=sp-01|pps_messagelog|
0 Karma
1 Solution

jeff
Contributor

Add the following to local/props.conf

[pps_messagelog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

[pps_maillog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

added 1 May 2019

Add the following to local/limits.conf

[spath]
extract_all=true
extraction_cutoff=30000

note: version 1.0.2 of the app added "DATETIME_CONFIG = CURRENT" to default/props.conf, which overrides these settings. I edited default to remove that configuration, but "proper" method would be to add "DATETIME_CONFIG = /etc/datetime.xml" to local/props.conf to restore the default. I have reported this to the developers and they responded they're workig on a revised version to include proper date/time extraction.

View solution in original post

jeff
Contributor

Add the following to local/props.conf

[pps_messagelog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

[pps_maillog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

added 1 May 2019

Add the following to local/limits.conf

[spath]
extract_all=true
extraction_cutoff=30000

note: version 1.0.2 of the app added "DATETIME_CONFIG = CURRENT" to default/props.conf, which overrides these settings. I edited default to remove that configuration, but "proper" method would be to add "DATETIME_CONFIG = /etc/datetime.xml" to local/props.conf to restore the default. I have reported this to the developers and they responded they're workig on a revised version to include proper date/time extraction.

ashutoshab
Communicator

I also had the same issue. I tried setting up timestamp configuration myself but i messed something. Later found that there was slight mistake in the Regex for TIME_PREFIX. I used your suggestion and now I am perfect.

Thanks.

Upvoted.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...