Getting Data In

Getting wrong timestamp in GC logs

AKG1_old1
Builder

Hello,

we have configured to pick time stamp from the logs itself but in some cases time stamp is not present. In this case Timestamp is assigned based on VM infor in GC logs.

**Logs with Timestamp**
2018-10-26T10:45:22.124+0200: 8372.847: [GC [1 CMS-initial-mark: 85682K(117800K)] 86784K(127592K), 0.0016081 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2018-10-26T10:47:25.532+0200: 8371.639: [GC [1 CMS-initial-mark: 65392K(117800K)] 66569K(127592K), 0.0016188 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 

**Logs without Time stamp**
69985.620: [GC [PSYoungGen: 18336K->1408K(18944K)] 31821K->14909K(45568K), 0.0322354 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
67891.316: [GC [PSYoungGen: 17024K->928K(18944K)] 30509K->14413K(45568K), 2094.1194859 secs] [Times: user=5859.78 sys=125.11, real=2094.12 secs] 

props.conf
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX = ^

In GC logs some time VM information is printed and if there is no timestamp it pick VM time eventhough VM time is not matching with props.conf . (See attachment)
In this case eventhough logs are generated today it's displaying in year 2015. Is it possible if no timestamp present, assign upload time as time stamp ?

See attachment.alt text

0 Karma
1 Solution

FrankVl
Ultra Champion

When Splunk cannot find a timestamp, it will take the timestamp from the previous event. In your case that has a bit of an odd side effect, since the first event without a timestamp, actually does contain a timestamp, but not a useful one.

I'm not sure how you could prevent Splunk from using that "built on" timestamp, such that already that event would get assigned a timestamp based on the previous event. Perhaps reducing the MAX_TIMESTAMP_LOOKAHEAD setting in props.conf could work. Setting that to 30 should be enough to capture the proper timestamps, and may (but I'm not 100% sure of the behavior here) prevent splunk from picking up that useless build timestamp further down the event.

View solution in original post

FrankVl
Ultra Champion

When Splunk cannot find a timestamp, it will take the timestamp from the previous event. In your case that has a bit of an odd side effect, since the first event without a timestamp, actually does contain a timestamp, but not a useful one.

I'm not sure how you could prevent Splunk from using that "built on" timestamp, such that already that event would get assigned a timestamp based on the previous event. Perhaps reducing the MAX_TIMESTAMP_LOOKAHEAD setting in props.conf could work. Setting that to 30 should be enough to capture the proper timestamps, and may (but I'm not 100% sure of the behavior here) prevent splunk from picking up that useless build timestamp further down the event.

AKG1_old1
Builder

Thanks !! this worked for me.

MAX_TIMESTAMP_LOOKAHEAD=25

I tried other options as well like removing the line with VM insformation using transform but didn't worked.

Cheers!!

0 Karma
Get Updates on the Splunk Community!

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...

Purpose in Action: How Splunk Is Helping Power an Inclusive Future for All

At Cisco, purpose isn’t a tagline—it’s a commitment. Cisco’s FY25 Purpose Report outlines how the company is ...

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...