Getting Data In

How do I configure Splunk to recognize the non-standard timestamp format in my log file?

markwymer
Path Finder

Hi All,

I'm trying to Upload a file/log using the 'Add Data' -> 'upload' in Splunk Web. Unfortunately, as per most of our logs, the input isn't in a structured format 😞

An example event looks like:-

my_application : access_live_05_6021 : 2015//08//18 20/:33/:24 Z : SUCCESS : apps.baplc.com%2Ftravel%2Fcarsproxy%2Fpublic%2Fen

My, initial, problem is that I can't get Splunk to recognise the timestamp - 2015//08//18 20/:33/:24 Z - I tried $Y//%m//%d $H/:$M/:%S Z in the 'Timestamp -> Advanced -> Timestamp Format' field, but it still couldn't detect the date field. I have a feeling that there is some kind of regex escape type stuff required, but ( I think ) I've tried everything except the correct solution!

The second question - for an extra bonus point 🙂 - is there an easy way in Splunk to change the apps.baplc.com%2Ftravel%2Fgeneral%2Fpublic%2Fen to apps.baplc.com/travel/general/public/en

Many thanks for any help,
Mark.

Tags (1)
0 Karma
1 Solution

somesoni2
Revered Legend

In Splunk Web -> Data Preview, In Timestamps tab, use following

1) Timestamp is always prefaced by a pattern - ^\s*\w+\s*:\s*\w+\s*:\s*
2) Timestamp format (strptime) - %Y//%m//%d %H/:%M/:%S

View solution in original post

0 Karma

markwymer
Path Finder

My apologies - I accidentally used a '$' instead of a '%' in my previous comment! It should have read...

"I tried $Y//%m//%d %H/:%M/:%S Z in the 'Timestamp -> Advanced -> Timestamp Format' field"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this in the Timestamp Format box:

%Y//%m//%d %H/:%M/:%S %Z

No escaping is necessary, unless you want to include the literal '%' character in your format string. If it doesn't work, try specifying :\s+ as the time prefix.

For your second question, consider added a sed command to your props.conf file:

[mysourcetype]
SEDCMD-slash = s/%2F/\//g
---
If this reply helps you, Karma would be appreciated.

somesoni2
Revered Legend

In Splunk Web -> Data Preview, In Timestamps tab, use following

1) Timestamp is always prefaced by a pattern - ^\s*\w+\s*:\s*\w+\s*:\s*
2) Timestamp format (strptime) - %Y//%m//%d %H/:%M/:%S

0 Karma

maciep
Champion

First off, are the dollar signs in your timestamp format typos or actually what you tried? They should be percent signs.

For the second part I think the urldecode function should work. As an example

| noop | stats count | eval blah="apps.baplc.com%2Ftravel%2Fgeneral%2Fpublic%2Fen"   | eval meh =urldecode(blah)
0 Karma

markwymer
Path Finder

my apologies - yes the '$' were a typo

0 Karma
Get Updates on the Splunk Community!

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...