Splunk Search

There are 2 timestamp formats in a log file

lish123
Loves-to-Learn Lots

<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794

Labels (1)
Tags (1)
0 Karma

to4kawa
Ultra Champion
index=_internal | head 1 | fields _raw _time
| eval _raw="<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794
<CST 6:58:06 PM, January 10, 2021> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794"
| multikv noheader=t
| fields - Col*

| rex "<(?P<time>.*?)>"
| eval _time=case(match(time,",\d{3}\s*[AP]M"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T,%3N %p %:z"),
match(time,"^[A-Z][a-z]+\b"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T %p %:z"),
match(time,"^CST+\b"),strptime(replace(time,"CST","-0600"),"%:z %T %p, %B %d, %Y"))

 

How about INGEST_EVAL?

https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/IngestEval

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot. @to4kawa

I try to run your spl statement, and the result is only 3 events, but I have more than 3 events. Can I provide the configuration props.conf file to extract the timestamp when uploading new data?

Tags (1)
0 Karma

to4kawa
Ultra Champion

Set up transforms.conf as described in the reference.

First, extract the time field with REGEX and then set INGEST_EVAL.
For eval, you can use the same one as in SPL.

0 Karma

General_Talos
Path Finder

Check in props.conf and adjust timezone settings with "TZ" for your sourcetype

Example

[your sourcetype]
TZ = GMT

https://community.splunk.com/t5/Archive/How-to-adjust-the-time-zone-for-an-logs-coming-into-splunk/m...

 

0 Karma

lish123
Loves-to-Learn Lots

It’s not a time zone issue. One event contains milliseconds and the other does not contain milliseconds.

0 Karma

askkawalkar
Path Finder

What is your requirement ? are you looking for ingesting these 2 events in splunk with 2 time format ? 
or you are looking to make constant time format for an all event when ingested to splunk .

I can try to provide solution once requirement are specified. 

Meanwhile for both type of question you will get answer in example of below documentation

0 Karma

lish123
Loves-to-Learn Lots

I want to extract these 2 events in a timestamp format.

0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

Please place below parameters in props.conf, replace SOURCETYPENAME with actual sourcetype.  If you are aware about regex, you can define your own regex

[SOURCETYPENAME]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = \<(?<Time>\w{3} \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2}[,\d{3}]* PM CST)\>
TIME_FORMAT = %b %d, %Y %I:%M:%S,%Q %p %Z

 

Regards. 

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot @askkawalkar

However, this TIME_FORMAT

%B%d,%Y%I:%M:%S,%Q%p%Z
Cannot extract this type of timestamp correctly

<CST 6:58:06 PM, January 10, 2021>

Tags (1)
0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

 

I have developed solution based on timestamp in sample data you provided.

Is there any more types of timeformat in the log ?

Regards.

0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...