Splunk Search

There are 2 timestamp formats in a log file

lish123
Loves-to-Learn Lots

<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794

Tags (1)
0 Karma

to4kawa
Ultra Champion
index=_internal | head 1 | fields _raw _time
| eval _raw="<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794
<CST 6:58:06 PM, January 10, 2021> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794"
| multikv noheader=t
| fields - Col*

| rex "<(?P<time>.*?)>"
| eval _time=case(match(time,",\d{3}\s*[AP]M"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T,%3N %p %:z"),
match(time,"^[A-Z][a-z]+\b"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T %p %:z"),
match(time,"^CST+\b"),strptime(replace(time,"CST","-0600"),"%:z %T %p, %B %d, %Y"))

 

How about INGEST_EVAL?

https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/IngestEval

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot. @to4kawa

I try to run your spl statement, and the result is only 3 events, but I have more than 3 events. Can I provide the configuration props.conf file to extract the timestamp when uploading new data?

Tags (1)
0 Karma

to4kawa
Ultra Champion

Set up transforms.conf as described in the reference.

First, extract the time field with REGEX and then set INGEST_EVAL.
For eval, you can use the same one as in SPL.

0 Karma

General_Talos
Path Finder

Check in props.conf and adjust timezone settings with "TZ" for your sourcetype

Example

[your sourcetype]
TZ = GMT

https://community.splunk.com/t5/Archive/How-to-adjust-the-time-zone-for-an-logs-coming-into-splunk/m...

 

0 Karma

lish123
Loves-to-Learn Lots

It’s not a time zone issue. One event contains milliseconds and the other does not contain milliseconds.

0 Karma

askkawalkar
Path Finder

What is your requirement ? are you looking for ingesting these 2 events in splunk with 2 time format ? 
or you are looking to make constant time format for an all event when ingested to splunk .

I can try to provide solution once requirement are specified. 

Meanwhile for both type of question you will get answer in example of below documentation

0 Karma

lish123
Loves-to-Learn Lots

I want to extract these 2 events in a timestamp format.

0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

Please place below parameters in props.conf, replace SOURCETYPENAME with actual sourcetype.  If you are aware about regex, you can define your own regex

[SOURCETYPENAME]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = \<(?<Time>\w{3} \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2}[,\d{3}]* PM CST)\>
TIME_FORMAT = %b %d, %Y %I:%M:%S,%Q %p %Z

 

Regards. 

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot @askkawalkar

However, this TIME_FORMAT

%B%d,%Y%I:%M:%S,%Q%p%Z
Cannot extract this type of timestamp correctly

<CST 6:58:06 PM, January 10, 2021>

Tags (1)
0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

 

I have developed solution based on timestamp in sample data you provided.

Is there any more types of timeformat in the log ?

Regards.

0 Karma
Get Updates on the Splunk Community!

Observability Unlocked: Kubernetes Monitoring with Splunk Observability Cloud

  Ready to master Kubernetes and cloud monitoring like the pros?Join Splunk’s Growth Engineering team for an ...

Wrapping Up Cybersecurity Awareness Month

October might be wrapping up, but for Splunk Education, cybersecurity awareness never goes out of season. ...

🌟 From Audit Chaos to Clarity: Welcoming Audit Trail v2

&#x1f5e3; You Spoke, We Listened  Audit Trail v2 wasn’t written in isolation—it was shaped by your voices.  In ...