Archive
Highlighted

How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

I am getting the below two warning messages,
1. 11-27-2017 06:00:22.902 +1100 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Mon Nov 27 06:00:20 2017). Context: source::C:\Program Files\Splunk\var\log\splunk\pdfgen.log|host::INDEXER|splunk_pdfgen|20662

11-27-2017 06:00:16.835 +1100 WARN LineBreakingProcessor - Truncating line because limit of 10000 bytes has been exceeded with a line length >= 17586 - datasource="C:\Program Files\Splunk\var\log\splunk\pdfgen.log", datahost="INDEXER", datasourcetype="splunkpdfgen"

  1. Sample timestamp in pdfgen.log looks like this
    2017-11-27 06:01:00,206 +1100 INFO pdfgentable:1041 - renderTable> headerRow: ['host', 'srcinterface', 'portstatus', 'count']
    2017-11-27 06:01:09,519 +1100 INFO pdfgen
    endpoint:271 - Generated pdf, filename = overview-2017-11-27.pdf

  2. props.conf
    [splunkpdfgen]
    TIME
    FORMAT = %m-%d-%Y %H:%M%S,%l
    SHOULDLINEMERGE = False
    MAX
    TIMESTAMP_LOOKAHEAD = 40

Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

SplunkTrust
SplunkTrust

Hi damode,

the TIME_FORMAT = %m-%d-%Y %H:%M%S,%l should be TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N.
Regarding the truncating add TRUNCATE = 20000 to the props.conf

Hope this helps ...

cheers, MuS

Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

Hi @MuS,

Thanks for your prompt reply.

I have applied the suggested settings. Will let you know the outcome.

Regards,
Dev

0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

Hi @MuS,

I am not getting Truncating line issue anymore. Thanks for that! I am still, however, getting the timestamp issues.

  1. 11-28-2017 06:00:16.854 +1100 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Nov 28 06:00:14 2017). Context: source::C:\Program Files\Splunk\var\log\splunk\pdfgen.log|host::INDEXER|splunk_pdfgen|126
  2. props.conf [splunkpdfgen] TIMEFORMAT = %Y-%m-%d %H:%M:%S,%3N SHOULDLINEMERGE = False MAXTIMESTAMP_LOOKAHEAD = 40 TRUNCATE = 20000
0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

SplunkTrust
SplunkTrust

I just checked the default settings for [splunk_pdfgen] and it actually has this option set:

 TIME_FORMAT = %m-%d-%Y %H:%M%S,%l

So, please remove the TIME_FORMAT you added and try again - really wired...

Can you run this command /opt/splunk/bin/splunk btool props list splunk_pdfgen --debug and compare to this list of options please:

/opt/splunk/etc/system/default/props.conf                  [splunk_pdfgen]
/opt/splunk/etc/system/default/props.conf                  ADD_EXTRA_TIME_FIELDS = True
/opt/splunk/etc/system/default/props.conf                  ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf                  AUTO_KV_JSON = true
/opt/splunk/etc/system/default/props.conf                  BREAK_ONLY_BEFORE = 
/opt/splunk/etc/system/default/props.conf                  BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf                  CHARSET = UTF-8
/opt/splunk/etc/system/default/props.conf                  DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf                  HEADER_MODE = 
/opt/splunk/etc/system/default/props.conf                  LEARN_MODEL = true
/opt/splunk/etc/system/default/props.conf                  LEARN_SOURCETYPE = true
/opt/splunk/etc/system/default/props.conf                  LINE_BREAKER_LOOKBEHIND = 100
/opt/splunk/etc/system/default/props.conf                  MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/props.conf                  MAX_DAYS_AGO = 2000
/opt/splunk/etc/system/default/props.conf                  MAX_DAYS_HENCE = 2
/opt/splunk/etc/system/default/props.conf                  MAX_DIFF_SECS_AGO = 3600
/opt/splunk/etc/system/default/props.conf                  MAX_DIFF_SECS_HENCE = 604800
/opt/splunk/etc/system/default/props.conf                  MAX_EVENTS = 256
/opt/splunk/etc/system/default/props.conf                  MAX_TIMESTAMP_LOOKAHEAD = 40
/opt/splunk/etc/system/default/props.conf                  MUST_BREAK_AFTER = 
/opt/splunk/etc/system/default/props.conf                  MUST_NOT_BREAK_AFTER = 
/opt/splunk/etc/system/default/props.conf                  MUST_NOT_BREAK_BEFORE = 
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION = indexing
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-all = full
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-inner = inner
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-outer = outer
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-raw = none
/opt/splunk/etc/system/default/props.conf                  SEGMENTATION-standard = standard
/opt/splunk/etc/system/default/props.conf                  SHOULD_LINEMERGE = False
/opt/splunk/etc/system/default/props.conf                  TIME_FORMAT = %m-%d-%Y %H:%M%S,%l
/opt/splunk/etc/system/default/props.conf                  TRANSFORMS = 
/opt/splunk/etc/system/default/props.conf                  TRUNCATE = 10000
/opt/splunk/etc/system/default/props.conf                  detect_trailing_nulls = false
/opt/splunk/etc/system/default/props.conf                  maxDist = 100
/opt/splunk/etc/system/default/props.conf                  priority = 
/opt/splunk/etc/system/default/props.conf                  sourcetype = 
0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

Hi @MuS,

Upon comparing with the above list of options, I found the below fields having different value in comparison to yours. Everything else is same.
CHARSET = AUTO
TRUNCATE = 20000
detecttrailingnulls = auto

0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

Hi @MuS, I had changed back to default TIME_FORMAT, but that still gave the same issue.
Based on the above observation, do you recommend setting the [splunk_pdfgen] attributes exactly same as yours ?

0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

SplunkTrust
SplunkTrust

Well, the above settings are the Splunk default settings so they really should work.

0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

Builder

Now I am getting the same error from datasourcetype = licensealert-5 as well, in addition to splunk_pdfgen.

0 Karma
Highlighted

Re: How to resolve timestamp and line processing issues in pdfgen.log ?

SplunkTrust
SplunkTrust

That sounds like a bigger problem here .... also reading all you other questions.

Random question: have you done a FS check lately on your Splunk server to see if everything is healthy?

0 Karma