Getting Data In

TimeFormat Error from a line in nullQueue

pshumate
Explorer

The transform works and filters out the the matching line from going into the index but I still get these errors:

WARN  DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event.  Context="source::/export/splunk/incoming/we_accesslog_extsqu_xxx.xxx.xxx.xxx_20120326_142201_32865.gz|host::xxx.xxx.xxx.xxx|cdsis-extended-squid|remoteport::38810" Text="#Number of transaction records: 1..."

I want to avoid the timestamp error while continuing to filter the headers and footers to nullQueue. Any suggestions?

Example Log

#Software: (CDS 2.6.1 b17)
Current-Time Time-to-Serve Client-IP Request-Desc/Status-Returned Bytes-Xferred Method URL MIME-Type
[21/Mar/2012:04:42:00.931+0000] 474623 xxx.xxx.xxx.xxx TCP_MISS/200 1807152 GET http://url.coms
[21/Mar/2012:04:42:01.275+0000] 323330 xxx.xxx.xxx.xxx TCP_MISS/200 1152750 GET http://url.coms
[21/Mar/2012:04:42:01.610+0000] 52900 xxx.xxx.xxx.xxx TCP_MISS/200 37486 GET http://url.coms
[21/Mar/2012:04:42:02.001+0000] 108528 xxx.xxx.xxx.xxx TCP_MISS/200 640556 GET http://url.coms
#Number of transaction records: 100

Props.conf

[source::...we_accesslog...]
TRANSFORMS-debug_log = debug_log_footer_trans, debug_log_header_trans

[cdsis-extended-squid]
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = false
# TIME_PREFIX = ^\[
# TIME_FORMAT = %d/%b/%Y:%H:%M:%S.%3N
REPORT-cdsis_ext = cdsis_ext_squid_transform
EXTRACT-duration = ^\[\d+/\w{3}/\d+\:\d{2}\:\d{2}\:\d{2}\.\d{3,}\+\d{3,}\]\s(?<duration>\d+)
KV_MODE = none
MAX_DAYS_AGO = 10

Transforms.conf

[debug_log_footer_trans]
REGEX=^.?Number
DEST_KEY = queue
FORMAT = nullQueue

[debug_log_header_trans]
REGEX=^Current-Time|^.Software
DEST_KEY = queue
FORMAT = nullQueue

[cdsis_ext_squid_transform]
REGEX =     ^\[\d+/\w{3}/\d+\:\d{2}\:\d{2}\:\d{2}\.\d{3,}\+\d{3,}\]\s(\d+)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(\w+)/(\d+)\s(\d+)\s(\w+)\s(http://\S+)\s(\w+/\w+)\s+$
FORMAT =    Duration::$1 ClientIP::$2 TCPStatus::$3 HTTPStatus::$4 BytesReturned::$5 HTTPMethod::$6 URL::$7 MimeType::$8
0 Karma

woodcock
Esteemed Legend

You need to tell Splunk something about how to timestamp your events. I see you have commented out these 2 lines (and that one is not quite right); I suggest you uncomment them and use these values:

TIME_PREFIX = ^\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S.%3N%Z

Alternatively, if you commented out these lines because the timestamps in the events are bogus, you should use one of these two settings:

DATETIME_CONFIG = CURRENT
DATETIME_CONFIG = NONE

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition#Timestamp_attr...

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas     Cisco Live 2026 is almost here, and this ...

Data Management Digest – May 2026

Welcome to the May 2026 edition of Data Management Digest!   As your trusted partner in data innovation, the ...