topic Re: Splunk Data Parse Verbose issue in Getting Data In

Splunk Data Parse Verbose issue

ram254481493 — Thu, 06 Jun 2019 22:37:57 GMT

Hi, Currently I am having below issues :

A possible timestamp match (Fri Aug 16 11:09:15 2013) is outside of the acceptable time window.
• Accepted time (Fri Apr 5 00:00:00 2019) is suspiciously far away from the previous event's time (Thu Jun 6 14:10:32 2019), but still acceptable because it was extracted by the same pattern Splunk
• Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of the event. Splunk
• Time parsed (Fri Apr 8 00:00:00 2016) is too far away from the previous event's time (Thu Jun 6 11:37:19 2019) to be accepted. If this is a correct time, MAX_DIFF_SECS_AGO (3600) or MAX_DIFF_SECS_HENCE Splunk.

Below is my props.conf deployed in indexers:

[xxxxxxxxx]
DATETIME_CONFIG = 
FIELD_DELIMITER = |
FIELD_NAMES = timestamp,message
HEADER_FIELD_DELIMITER = |
INDEXED_EXTRACTIONS = psv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Pipe-separated value format. 
disabled = false
pulldown_type = true
SEDCMD-stupid-line-breaker = s/\----------------------------------------//g

I am planning to add these settings on my existing props configs ;

• Add MAX_DAYS_AGO  = 3000
• MAX_DAYS_HENCE = 90

Will it solve my issue if I add the above two settings?
I know by default MAX_DAYS_AGO is 2000 but here nothing is mentioned means is the default of 2000 days working?

What stanza do I need to add for this error message Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of the event?

Please suggest some answers.

Re: Splunk Data Parse Verbose issue

richgalloway — Thu, 06 Jun 2019 22:53:28 GMT

Please share some sample data. It's close to impossible to determine if your settings are correct without knowing what is to parse.

BTW, the SEDCMD attribute goes in transforms.conf.

Re: Splunk Data Parse Verbose issue

ram254481493 — Thu, 06 Jun 2019 23:05:20 GMT

Sure here is sample event logs from splunk , i dont have actual logs now , if it wont works i will upload the actual logs shortly :

6/6/19
5:57:31.000 PM

6/6/2019 5:57:31 PM|Thread Id: 10252|Information|Information|xxxxxxxxxx|Ended UnemploymentDocumentProcessor Main Total Time: 00:03:42.5002966

host = xxxxxxxxxxxx source = \xxxxxxxxxxx\Logs\Prod\PlatformServices\UnemploymentDocumentProcessorClaimsQuestionaires\Audit.log sourcetype = xxxxxxxxxx
6/6/19
5:57:31.000 PM

6/6/2019 5:57:31 PM|Thread Id: 10252|Information|Information|xxxxxxxxxxxx|End time: 6/6/2019 5:57:31 PM Time Taken: 00:00:07.5917984 for validation work Reference: 502199417

host = xxxxxxxxxxxxx source = \xxxxxxxxxxxxxx\Logs\Prod\PlatformServices\UnemploymentDocumentProcessorClaimsQuestionaires\Audit.log sourcetype = xxxxxxxxxxx

Re: Splunk Data Parse Verbose issue

ram254481493 — Fri, 07 Jun 2019 14:11:56 GMT

here is the sample of actual server logs:

06/05/2019 11:56:53.86484|xxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeSpecificationFactory.CreateSpecification(Input Parameters Ignored)|

06/05/2019 11:56:53.88046|xxxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeLegacyDao.GetChargeServiceStatus(Output Parameters Ignored) -> 1 : TimeTaken -31.2485ms|

06/05/2019 11:56:53.98987|xxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeSpecificationFactory.CreateSpecification(Output Parameters Ignored) -> UCM.Charges.BusinessLayerImplementation.Specifications.PendingHearingLevelSpecification : TimeTaken -0ms|

06/05/2019 11:56:53.98987|xxxxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeSpecificationFactory.CreateSpecification(Output Parameters Ignored) -> UCM.Charges.BusinessLayerImplementation.Specifications.ReimbursableSpecification : TimeTaken -0ms|

06/05/2019 11:56:54.02108|xxxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeSpecificationFactory.CreateSpecification(Input Parameters Ignored)|

06/05/2019 11:56:54.02108|xxxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeSpecificationFactory.CreateSpecification(Output Parameters Ignored) -> UCM.Charges.BusinessLayerImplementation.Specifications.UnfavorableDeterminationUnderAppealSpecification : TimeTaken -0ms|

06/05/2019 11:56:54.05234|xxxxxxxxxxx|/LM/W3SVC/10/ROOT/ChargeService-1-132041830473540422|IChargeVerificationRules.VerifyIsReimbursible(Output Parameters Ignored) -> False : TimeTaken -187.5012ms|

Please help me out to fix these above mentioned errors.

Re: Splunk Data Parse Verbose issue

richgalloway — Fri, 07 Jun 2019 15:39:31 GMT

I'm not sure where the Aug 16 date is coming from, but it should help to specify timestamp options. Also, INDEXED_EXTRACTIONS = psv doesn't apply here since you don't have a PSV file. Try these settings:

[mysecretsourcetype]
# Break after a line of dashes and a CR and/or LF.  Discard the matching chars.
LINE_BREAKER = (----+[\r\n]+)
TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%Y %H:%M:%S.%5N
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false

The "Failed to parse timestamp" message may be coming from the lines of dashes. These settings should eliminate those.

Re: Splunk Data Parse Verbose issue

ram254481493 — Wed, 30 Sep 2020 00:51:47 GMT

so i will remove the line INDEXED_EXTRACTIONS = psv and adding these lines to my existing sourcetype right ? and will it fix these issues too or should i need to add some extra parameter like (• Add MAX_DAYS_AGO = 3000
• MAX_DAYS_HENCE = 90)

A possible timestamp match (Fri Aug 16 11:09:15 2013) is outside of the acceptable time window.
• Accepted time (Fri Apr 5 00:00:00 2019) is suspiciously far away from the previous event's time (Thu Jun 6 14:10:32 2019), but still accepted because it was extracted by the same pattern splunk
Time parsed (Fri Apr 8 00:00:00 2016) is too far away from the previous event's time (Thu Jun 6 11:37:19 2019) to be accepted. If this is a correct time, MAX_DIFF_SECS_AGO (3600) or MAX_DIFF_SECS_HENCE splunk