Getting Data In

TimeFormat Error from a line in nullQueue

pshumate
Explorer

The transform works and filters out the the matching line from going into the index but I still get these errors:

WARN  DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event.  Context="source::/export/splunk/incoming/we_accesslog_extsqu_xxx.xxx.xxx.xxx_20120326_142201_32865.gz|host::xxx.xxx.xxx.xxx|cdsis-extended-squid|remoteport::38810" Text="#Number of transaction records: 1..."

I want to avoid the timestamp error while continuing to filter the headers and footers to nullQueue. Any suggestions?

Example Log

#Software: (CDS 2.6.1 b17)
Current-Time Time-to-Serve Client-IP Request-Desc/Status-Returned Bytes-Xferred Method URL MIME-Type
[21/Mar/2012:04:42:00.931+0000] 474623 xxx.xxx.xxx.xxx TCP_MISS/200 1807152 GET http://url.coms
[21/Mar/2012:04:42:01.275+0000] 323330 xxx.xxx.xxx.xxx TCP_MISS/200 1152750 GET http://url.coms
[21/Mar/2012:04:42:01.610+0000] 52900 xxx.xxx.xxx.xxx TCP_MISS/200 37486 GET http://url.coms
[21/Mar/2012:04:42:02.001+0000] 108528 xxx.xxx.xxx.xxx TCP_MISS/200 640556 GET http://url.coms
#Number of transaction records: 100

Props.conf

[source::...we_accesslog...]
TRANSFORMS-debug_log = debug_log_footer_trans, debug_log_header_trans

[cdsis-extended-squid]
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = false
# TIME_PREFIX = ^\[
# TIME_FORMAT = %d/%b/%Y:%H:%M:%S.%3N
REPORT-cdsis_ext = cdsis_ext_squid_transform
EXTRACT-duration = ^\[\d+/\w{3}/\d+\:\d{2}\:\d{2}\:\d{2}\.\d{3,}\+\d{3,}\]\s(?<duration>\d+)
KV_MODE = none
MAX_DAYS_AGO = 10

Transforms.conf

[debug_log_footer_trans]
REGEX=^.?Number
DEST_KEY = queue
FORMAT = nullQueue

[debug_log_header_trans]
REGEX=^Current-Time|^.Software
DEST_KEY = queue
FORMAT = nullQueue

[cdsis_ext_squid_transform]
REGEX =     ^\[\d+/\w{3}/\d+\:\d{2}\:\d{2}\:\d{2}\.\d{3,}\+\d{3,}\]\s(\d+)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(\w+)/(\d+)\s(\d+)\s(\w+)\s(http://\S+)\s(\w+/\w+)\s+$
FORMAT =    Duration::$1 ClientIP::$2 TCPStatus::$3 HTTPStatus::$4 BytesReturned::$5 HTTPMethod::$6 URL::$7 MimeType::$8
0 Karma

woodcock
Esteemed Legend

You need to tell Splunk something about how to timestamp your events. I see you have commented out these 2 lines (and that one is not quite right); I suggest you uncomment them and use these values:

TIME_PREFIX = ^\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S.%3N%Z

Alternatively, if you commented out these lines because the timestamps in the events are bogus, you should use one of these two settings:

DATETIME_CONFIG = CURRENT
DATETIME_CONFIG = NONE

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition#Timestamp_attr...

0 Karma
Get Updates on the Splunk Community!

App Platform's 2025 Year in Review: A Year of Innovation, Growth, and Community

As we step into 2026, it’s the perfect moment to reflect on what an extraordinary year 2025 was for the Splunk ...

Operationalizing Entity Risk Score with Enterprise Security 8.3+

Overview Enterprise Security 8.3 introduces a powerful new feature called “Entity Risk Scoring” (ERS) for ...

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...