All Apps and Add-ons

timestamp not being recognized for Proofpoint Add-On

jeff
Contributor

Timestamps are not properly being parsed using v1.0.0 of the TA-pps_ondemand app.

01-16-2019 02:56:09.819 -0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Wed Jan 16 02:28:43 2019). Context: source=proofpoint_message_log|host=sp-01|pps_messagelog|
0 Karma
1 Solution

jeff
Contributor

Add the following to local/props.conf

[pps_messagelog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

[pps_maillog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

added 1 May 2019

Add the following to local/limits.conf

[spath]
extract_all=true
extraction_cutoff=30000

note: version 1.0.2 of the app added "DATETIME_CONFIG = CURRENT" to default/props.conf, which overrides these settings. I edited default to remove that configuration, but "proper" method would be to add "DATETIME_CONFIG = /etc/datetime.xml" to local/props.conf to restore the default. I have reported this to the developers and they responded they're workig on a revised version to include proper date/time extraction.

View solution in original post

jeff
Contributor

Add the following to local/props.conf

[pps_messagelog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

[pps_maillog]
LINE_BREAKER            = ([\r\n]+\s*)\{
TIME_PREFIX             = \"ts\"\s*:\s*\"
MAX_TIMESTAMP_LOOKAHEAD = 34
TIME_FORMAT             = %Y-%m-%dT%H:%M:%S.%6N%z

added 1 May 2019

Add the following to local/limits.conf

[spath]
extract_all=true
extraction_cutoff=30000

note: version 1.0.2 of the app added "DATETIME_CONFIG = CURRENT" to default/props.conf, which overrides these settings. I edited default to remove that configuration, but "proper" method would be to add "DATETIME_CONFIG = /etc/datetime.xml" to local/props.conf to restore the default. I have reported this to the developers and they responded they're workig on a revised version to include proper date/time extraction.

ashutoshab
Communicator

I also had the same issue. I tried setting up timestamp configuration myself but i messed something. Later found that there was slight mistake in the Regex for TIME_PREFIX. I used your suggestion and now I am perfect.

Thanks.

Upvoted.

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...