topic Re: How to resolve timestamp and line processing issues in pdfgen.log ? in Getting Data In

How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 29 Sep 2020 16:57:02 GMT

I am getting the below two warning messages,
1. 11-27-2017 06:00:22.902 +1100 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Mon Nov 27 06:00:20 2017). Context: source::C:\Program Files\Splunk\var\log\splunk\pdfgen.log|host::INDEXER|splunk_pdfgen|20662

11-27-2017 06:00:16.835 +1100 WARN LineBreakingProcessor - Truncating line because limit of 10000 bytes has been exceeded with a line length >= 17586 - data_source="C:\Program Files\Splunk\var\log\splunk\pdfgen.log", data_host="INDEXER", data_sourcetype="splunk_pdfgen"

Sample timestamp in pdfgen.log looks like this
2017-11-27 06:01:00,206 +1100 INFO pdfgen_table:1041 - renderTable> headerRow: ['host', 'src_interface', 'port_status', 'count']
2017-11-27 06:01:09,519 +1100 INFO pdfgen_endpoint:271 - Generated pdf, filename = overview-2017-11-27.pdf
props.conf
[splunk_pdfgen]
TIME_FORMAT = %m-%d-%Y %H:%M%S,%l
SHOULD_LINEMERGE = False
MAX_TIMESTAMP_LOOKAHEAD = 40

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

MuS — Mon, 27 Nov 2017 00:27:55 GMT

Hi damode,

the TIME_FORMAT = %m-%d-%Y %H:%M%S,%l should be TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N.
Regarding the truncating add TRUNCATE = 20000 to the props.conf

Hope this helps ...

cheers, MuS

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Mon, 27 Nov 2017 01:26:10 GMT

Hi @MuS,

Thanks for your prompt reply.

I have applied the suggested settings. Will let you know the outcome.

Regards,
Dev

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 29 Sep 2020 16:58:00 GMT

Hi @MuS,

I am not getting Truncating line issue anymore. Thanks for that! I am still, however, getting the timestamp issues.

11-28-2017 06:00:16.854 +1100 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Nov 28 06:00:14 2017). Context: source::C:\Program Files\Splunk\var\log\splunk\pdfgen.log|host::INDEXER|splunk_pdfgen|126
props.conf [splunk_pdfgen] TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N SHOULD_LINEMERGE = False MAX_TIMESTAMP_LOOKAHEAD = 40 TRUNCATE = 20000

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

MuS — Tue, 28 Nov 2017 00:49:29 GMT

I just checked the default settings for [splunk_pdfgen] and it actually has this option set:

 TIME_FORMAT = %m-%d-%Y %H:%M%S,%l

So, please remove the TIME_FORMAT you added and try again - really wired...

Can you run this command /opt/splunk/bin/splunk btool props list splunk_pdfgen --debug and compare to this list of options please:

/opt/splunk/etc/system/default/props.conf                  [splunk_pdfgen]
/opt/splunk/etc/system/default/props.conf                  ADD_EXTRA_TIME_FIELDS = True
/opt/splunk/etc/system/default/props.conf                  ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf                  AUTO_KV_JSON = true
/opt/splunk/etc/system/default/props.conf                  BREAK_ONLY_BEFORE = 
/opt/splunk/etc/system/default/props.conf                  BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf                  CHARSET = UTF-8
/opt/splunk/etc/system/default/props.conf                  DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf                  HEADER_MODE = 
/opt/splunk/etc/system/default/props.conf                  LEARN_MODEL = true
/opt/splunk/etc/system/default/props.conf                  LEARN_SOURCETYPE = true
/opt/splunk/etc/system/default/props.conf                  LINE_BREAKER_LOOKBEHIND = 100
/opt/splunk/etc/system/default/props.conf                  MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/props.conf                  MAX_DAYS_AGO = 2000
/opt/splunk/etc/system/default/props.conf                  MAX_DAYS_HENCE = 2
/opt/splunk/etc/system/default/props.conf                  MAX_DIFF_SECS_AGO = 3600
/opt/splunk/etc/system/default/props.conf                  MAX_DIFF_SECS_HENCE = 604800
/opt/splunk/etc/system/default/props.conf                  MAX_EVENTS = 256
/opt/splunk/etc/system/default/props.conf                  MAX_TIMESTAMP_LOOKAHEAD = 40
/opt/splunk/etc/system/default/props.conf                  MUST_BREAK_AFTER = 
/opt/splunk/etc/system/default/props.conf                  MUST_NOT_BREAK_AFTER = 
/opt/splunk/etc/system/default/props.conf                  MUST_NOT_BREAK_BEFORE = 
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION = indexing
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-all = full
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-inner = inner
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-outer = outer
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-raw = none
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-standard = standard
/opt/splunk/etc/system/default/props.conf                  SHOULD_LINEMERGE = False
/opt/splunk/etc/system/default/props.conf                  TIME_FORMAT = %m-%d-%Y %H:%M%S,%l
/opt/splunk/etc/system/default/props.conf                  TRANSFORMS = 
/opt/splunk/etc/system/default/props.conf                  TRUNCATE = 10000
/opt/splunk/etc/system/default/props.conf                  detect_trailing_nulls = false
/opt/splunk/etc/system/default/props.conf                  maxDist = 100
/opt/splunk/etc/system/default/props.conf                  priority = 
/opt/splunk/etc/system/default/props.conf                  sourcetype =

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 29 Sep 2020 16:59:13 GMT

Hi @MuS,

Upon comparing with the above list of options, I found the below fields having different value in comparison to yours. Everything else is same.
CHARSET = AUTO
TRUNCATE = 20000
detect_trailing_nulls = auto

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Mon, 04 Dec 2017 01:42:29 GMT

Hi @MuS, I had changed back to default TIME_FORMAT, but that still gave the same issue.
Based on the above observation, do you recommend setting the [splunk_pdfgen] attributes exactly same as yours ?

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

MuS — Mon, 04 Dec 2017 02:36:13 GMT

Well, the above settings are the Splunk default settings so they really should work.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 05 Dec 2017 01:30:56 GMT

Now I am getting the same error from datasourcetype = licensealert-5 as well, in addition to splunk_pdfgen.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

MuS — Tue, 05 Dec 2017 01:41:57 GMT

That sounds like a bigger problem here .... also reading all you other questions.

Random question: have you done a FS check lately on your Splunk server to see if everything is healthy?

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 05 Dec 2017 01:51:20 GMT

If you mean health check on DMC, then yes.
On Search head, I have license warning and scheduled searches skipped messages. On Indexer, I am getting these event processing issue about which I have posted here.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

MuS — Tue, 05 Dec 2017 01:54:48 GMT

No I meant an actual file system check from the operating system.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Tue, 05 Dec 2017 02:04:05 GMT

I just did a file system check from the operating system using SFC.EXE /scannow and did not find any integrity violations.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

damode — Wed, 13 Dec 2017 23:37:18 GMT

Hi @MuS, for some reason, the Search Head had the same hostname as the Indexer. Not sure how and when I did that. Once I changed it to its correct username, I stopped getting time parsing warning messages. I believe, that’s probably what was causing the issue.

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

arekdabrowski — Wed, 30 Sep 2020 03:43:49 GMT

I have the same problem on version 7.3.1
When I have the default props.conf file in the pdfgen file, my data quality displays problems with timestamp analysis, here are the details:
01-15-2020 11:56:18.641 +0100 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (40) characters of event. Defaulting to timestamp of previous event (Wed Jan 15 11:56:15 2020). Context: source=/opt/splunk/var/log/splunk/pdfgen.log|host=xxxxxxxxxxxxx|splunk_pdfgen|2557
When I add to my props.conf on the system / local / props.conf TIME_FORMAT =% Y-% m-% d% H:% M:% S,% 3N% z
I also have the same problem.
Do you have any ideas?