Getting Data In

How do I configure Splunk to recognize the non-standard timestamp format in my log file?

markwymer
Path Finder

Hi All,

I'm trying to Upload a file/log using the 'Add Data' -> 'upload' in Splunk Web. Unfortunately, as per most of our logs, the input isn't in a structured format 😞

An example event looks like:-

my_application : access_live_05_6021 : 2015//08//18 20/:33/:24 Z : SUCCESS : apps.baplc.com%2Ftravel%2Fcarsproxy%2Fpublic%2Fen

My, initial, problem is that I can't get Splunk to recognise the timestamp - 2015//08//18 20/:33/:24 Z - I tried $Y//%m//%d $H/:$M/:%S Z in the 'Timestamp -> Advanced -> Timestamp Format' field, but it still couldn't detect the date field. I have a feeling that there is some kind of regex escape type stuff required, but ( I think ) I've tried everything except the correct solution!

The second question - for an extra bonus point 🙂 - is there an easy way in Splunk to change the apps.baplc.com%2Ftravel%2Fgeneral%2Fpublic%2Fen to apps.baplc.com/travel/general/public/en

Many thanks for any help,
Mark.

Tags (1)
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

In Splunk Web -> Data Preview, In Timestamps tab, use following

1) Timestamp is always prefaced by a pattern - ^\s*\w+\s*:\s*\w+\s*:\s*
2) Timestamp format (strptime) - %Y//%m//%d %H/:%M/:%S

View solution in original post

0 Karma

markwymer
Path Finder

My apologies - I accidentally used a '$' instead of a '%' in my previous comment! It should have read...

"I tried $Y//%m//%d %H/:%M/:%S Z in the 'Timestamp -> Advanced -> Timestamp Format' field"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this in the Timestamp Format box:

%Y//%m//%d %H/:%M/:%S %Z

No escaping is necessary, unless you want to include the literal '%' character in your format string. If it doesn't work, try specifying :\s+ as the time prefix.

For your second question, consider added a sed command to your props.conf file:

[mysourcetype]
SEDCMD-slash = s/%2F/\//g
---
If this reply helps you, Karma would be appreciated.

somesoni2
SplunkTrust
SplunkTrust

In Splunk Web -> Data Preview, In Timestamps tab, use following

1) Timestamp is always prefaced by a pattern - ^\s*\w+\s*:\s*\w+\s*:\s*
2) Timestamp format (strptime) - %Y//%m//%d %H/:%M/:%S

0 Karma

maciep
Champion

First off, are the dollar signs in your timestamp format typos or actually what you tried? They should be percent signs.

For the second part I think the urldecode function should work. As an example

| noop | stats count | eval blah="apps.baplc.com%2Ftravel%2Fgeneral%2Fpublic%2Fen"   | eval meh =urldecode(blah)
0 Karma

markwymer
Path Finder

my apologies - yes the '$' were a typo

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...