topic Re: Timestamp parsing Failing !!! pls help in Getting Data In

Timestamp parsing Failing !!! pls help

xbbj3nj — Tue, 29 Apr 2014 18:46:11 GMT

Hi All,

All of a sudden, Timestamp parsing doesn't work in splunk when I index a file manually into the system. It ignores the logfile time and takes the current system time.
It was working well and I don't know suddenly what caused this problem, It's not even able to recognize the earlier indexed files.
It just gives the error "Timestamp parsing failed"..

Same case for event breaking too, it doesn't work either...

Can you please post a resolution to this ?

Re: Timestamp parsing Failing !!! pls help

linu1988 — Tue, 29 Apr 2014 19:26:12 GMT

sample logs and props.conf setting please

Re: Timestamp parsing Failing !!! pls help

xbbj3nj — Mon, 28 Sep 2020 16:29:48 GMT

Below is the props.conf under /opt/splunk/etc/system/default

[default]
CHARSET = UTF-8
LINE_BREAKER_LOOKBEHIND = 100
TRUNCATE = 10000
DATETIME_CONFIG = /etc/datetime.xml
ANNOTATE_PUNCT = True
HEADER_MODE =
MAX_DAYS_HENCE=2
MAX_DAYS_AGO=2000
MAX_DIFF_SECS_AGO=3600
MAX_DIFF_SECS_HENCE=604800
MAX_TIMESTAMP_LOOKAHEAD = 128
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
MAX_EVENTS = 256
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
TRANSFORMS =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner

Re: Timestamp parsing Failing !!! pls help

xbbj3nj — Mon, 28 Sep 2020 16:29:51 GMT

SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
LEARN_SOURCETYPE = true
maxDist = 100
detect_trailing_nulls = false

Re: Timestamp parsing Failing !!! pls help

xbbj3nj — Tue, 29 Apr 2014 19:40:44 GMT

sample log file :

Remark: DCS-p

Software: WebTrends SmartSource Data Collector

Version: 1.0

Date: 2014-04-22 04:47:49

Fields: date time c-ip cs-username cs-host cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-version cs(User-Agent) cs(Cookie) cs(Referer) dcs-id

2014-04-25 23:31:19 172.24.32.95 xbbjnzp fma.abc.net GET /fma/default.aspx
2014-04-25 23:31:31 172.24.32.95 xbbjnzp fma.abc.net GET /fma/default.aspx
2014-04-25 23:31:37 172.24.32.95 xbbjnzp fma.abc.net GET /fma/futures/default.aspx
2014-04-25 23:31:53 172.24.32.95 xbbjnzp fma.abc.net GET /fma/trades/default.aspx

Re: Timestamp parsing Failing !!! pls help

linu1988 — Tue, 29 Apr 2014 20:06:18 GMT

where exactly the log starts and ends? the config doesn't look right where these may parameters are mentioned!

simple

SHOULD_LINEMERGE=TRUE TIME_FORMAT=%Y-%m-%d %H:%M:%S BREAK_ONLY_BEFORE_DATE=TRUE

should see all the log times.

Re: Timestamp parsing Failing !!! pls help

somesoni2 — Tue, 29 Apr 2014 20:08:53 GMT

When you import a file into Splunk, you need to specify a sourcetype and configure sourcetype to correctly identify event breaking and timestamp. Till than if the data format is not in Splunk Standard (start with timestamp) it will show that error in preview screen.

Re: Timestamp parsing Failing !!! pls help

lguinn2 — Sat, 03 May 2014 10:03:08 GMT

So the actual log file looks more like this

#Remark: DCS-p
#Software: WebTrends SmartSource Data Collector     
#Version: 1.0
#Date: 2014-04-22 04:47:49
#Fields: date time c-ip cs-username cs-host cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-version cs(User-Agent) cs(Cookie) cs(Referer) dcs-id 
2014-04-25 23:31:19 172.24.32.95 xbbjnzp fma.abc.net GET /fma/default.aspx 
2014-04-25 23:31:31 172.24.32.95 xbbjnzp fma.abc.net GET /fma/default.aspx 
2014-04-25 23:31:37 172.24.32.95 xbbjnzp fma.abc.net GET /fma/futures/default.aspx 
2014-04-25 23:31:53 172.24.32.95 xbbjnzp fma.abc.net GET /fma/trades/default.aspx

Therefore, I believe that you will want the following stanza in a local props.conf - perhaps .../etc/system/local/props.conf

[yoursourcetype]
SHOULD_LINEMERGE = false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
PREAMBLE_REGEX = \#

The last line tells Splunk to ignore lines in the log file that begin with #. If you want to index these lines, just leave it off.

Make sure you specify the proper sourcetype for your inputs. If this problem is affecting multiple sourcetypes, add (or edit) a stanza in props.conf for each sourcetype.

I suspect that Splunk has begun to try to process the header, and that is confusing things.

Re: Timestamp parsing Failing !!! pls help

spammenot66 — Thu, 17 Mar 2016 16:48:46 GMT

are you manually importing Webtrends SDC log files into SPLUNK? If so, try using the SPLUNK forwarder.