Splunk Search

There are 2 timestamp formats in a log file

lish123
Loves-to-Learn Lots

<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794

Labels (1)
Tags (1)
0 Karma

to4kawa
Ultra Champion
index=_internal | head 1 | fields _raw _time
| eval _raw="<Jan 10, 2021 6:58:06 PM CST> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>
<Jan 10, 2021 6:58:06,538 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794
<CST 6:58:06 PM, January 10, 2021> <Notice> <Log Management> <BEA-170019> <The server log file weblogic.logging.FileStreamHandler instance=1128635794"
| multikv noheader=t
| fields - Col*

| rex "<(?P<time>.*?)>"
| eval _time=case(match(time,",\d{3}\s*[AP]M"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T,%3N %p %:z"),
match(time,"^[A-Z][a-z]+\b"),strptime(replace(time,"CST","-0600"),"%b %d, %Y %T %p %:z"),
match(time,"^CST+\b"),strptime(replace(time,"CST","-0600"),"%:z %T %p, %B %d, %Y"))

 

How about INGEST_EVAL?

https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/IngestEval

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot. @to4kawa

I try to run your spl statement, and the result is only 3 events, but I have more than 3 events. Can I provide the configuration props.conf file to extract the timestamp when uploading new data?

Tags (1)
0 Karma

to4kawa
Ultra Champion

Set up transforms.conf as described in the reference.

First, extract the time field with REGEX and then set INGEST_EVAL.
For eval, you can use the same one as in SPL.

0 Karma

General_Talos
Path Finder

Check in props.conf and adjust timezone settings with "TZ" for your sourcetype

Example

[your sourcetype]
TZ = GMT

https://community.splunk.com/t5/Archive/How-to-adjust-the-time-zone-for-an-logs-coming-into-splunk/m...

 

0 Karma

lish123
Loves-to-Learn Lots

It’s not a time zone issue. One event contains milliseconds and the other does not contain milliseconds.

0 Karma

askkawalkar
Path Finder

What is your requirement ? are you looking for ingesting these 2 events in splunk with 2 time format ? 
or you are looking to make constant time format for an all event when ingested to splunk .

I can try to provide solution once requirement are specified. 

Meanwhile for both type of question you will get answer in example of below documentation

0 Karma

lish123
Loves-to-Learn Lots

I want to extract these 2 events in a timestamp format.

0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

Please place below parameters in props.conf, replace SOURCETYPENAME with actual sourcetype.  If you are aware about regex, you can define your own regex

[SOURCETYPENAME]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = \<(?<Time>\w{3} \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2}[,\d{3}]* PM CST)\>
TIME_FORMAT = %b %d, %Y %I:%M:%S,%Q %p %Z

 

Regards. 

0 Karma

lish123
Loves-to-Learn Lots

Thanks a lot @askkawalkar

However, this TIME_FORMAT

%B%d,%Y%I:%M:%S,%Q%p%Z
Cannot extract this type of timestamp correctly

<CST 6:58:06 PM, January 10, 2021>

Tags (1)
0 Karma

askkawalkar
Path Finder

Hi @lish123 ,

 

I have developed solution based on timestamp in sample data you provided.

Is there any more types of timeformat in the log ?

Regards.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...