Splunk Search

There are 2 timestamp formats in a log file

lish123
Loves-to-Learn Lots

<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794

Labels (1)
Tags (1)
0 Karma

to4kawa
Ultra Champion
index=_internal | head 1 | fields _raw _time
| eval _raw="<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794
<CST 6:58:06 PM, January 10, 2021> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794"
| multikv noheader=t
| fields - Col*

| rex "<(?P<time>.*?)>"
| eval _time=case(match(time,",\d{3}\s*[AP]M"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T,%3N %p %:z"),
match(time,"^[A-Z][a-z]+\b"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T %p %:z"),
match(time,"^CST+\b"),strptime(replace(time,"CST","-0600"),"%:z %T %p, %B %d, %Y"))

 

How about INGEST_EVAL?

https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/IngestEval

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot. @to4kawa

I try to run your spl statement, and the result is only 3 events, but I have more than 3 events. Can I provide the configuration props.conf file to extract the timestamp when uploading new data?

Tags (1)
0 Karma

to4kawa
Ultra Champion

Set up transforms.conf as described in the reference.

First, extract the time field with REGEX and then set INGEST_EVAL.
For eval, you can use the same one as in SPL.

0 Karma

General_Talos
Path Finder

Check in props.conf and adjust timezone settings with "TZ" for your sourcetype

Example

[your sourcetype]
TZ = GMT

https://community.splunk.com/t5/Archive/How-to-adjust-the-time-zone-for-an-logs-coming-into-splunk/m...

 

0 Karma

lish123
Loves-to-Learn Lots

It’s not a time zone issue. One event contains milliseconds and the other does not contain milliseconds.

0 Karma

askkawalkar
Path Finder

What is your requirement ? are you looking for ingesting these 2 events in splunk with 2 time format ? 
or you are looking to make constant time format for an all event when ingested to splunk .

I can try to provide solution once requirement are specified. 

Meanwhile for both type of question you will get answer in example of below documentation

0 Karma

lish123
Loves-to-Learn Lots

I want to extract these 2 events in a timestamp format.

0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

Please place below parameters in props.conf, replace SOURCETYPENAME with actual sourcetype.  If you are aware about regex, you can define your own regex

[SOURCETYPENAME]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = \<(?<Time>\w{3} \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2}[,\d{3}]* PM CST)\>
TIME_FORMAT = %b %d, %Y %I:%M:%S,%Q %p %Z

 

Regards. 

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot @askkawalkar

However, this TIME_FORMAT

%B%d,%Y%I:%M:%S,%Q%p%Z
Cannot extract this type of timestamp correctly

<CST 6:58:06 PM, January 10, 2021>

Tags (1)
0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

 

I have developed solution based on timestamp in sample data you provided.

Is there any more types of timeformat in the log ?

Regards.

0 Karma
Get Updates on the Splunk Community!

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Combine Multiline Logs into a Single Event with SOCK: a Step-by-Step Guide for ...

Combine multiline logs into a single event with SOCK - a step-by-step guide for newbies Olga Malita The ...