Splunk Search

Can someone help in how _time is derived and how to derive correct ingested timestamp?

Karthikeyan
Engager

Hi Team,

I am ingesting job logs to SPlunk and below is one of the job log (job ran on 27th June) which was ingested with wrong _time value to SPlunk.

Job log:

(14.2) 06-27-22 10:31:03 (35312:24804)  PRINTFN: 2022.06.25
(14.2) 06-27-22 10:31:10 (35312:24804)      JOB: Job <ALERTS_MORNING> is completed successfully.

As the job ran on 27th June, the _time value in Splunk is showing as 25th June (hope it is derived from printfn in the logs). The date_mday field under _time is showing as 25 instead of 27.

Can someone help in how _time is derived (it is the ingested timestamp, but in this case it was calculated wrongly) and how to dervie correct ingested timestamp.

Regards, Karthikeyan.SV

Labels (2)
0 Karma

Karthikeyan
Engager

All,

Attached the entire job log file. Below are the props and transform.conf properties that is set for this sourcetype.

Props.conf
sourcetype = job_logs
[source::*\trace*.txt]
TRANSFORMS-set= setnull,setparsing


Transform.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = "^[^ \n]* (?P<Time>\d+\-\d+\-\d+\s+\d+:\d+:\d+) [^\n]\d+:\d+.\s+\w+: \w+ [^\w](?<Job>\w+)>.\w+.\w+.*"
DEST_KEY = queue
FORMAT = indexQueue

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The props.conf file is missing some important settings.  Splunk recommends all sourcetypes have at least these 8 settings: LINE_BREAKER, TIME_PREFIX, TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, TRUNCATE, SHOULD_LINEMERGE, EVENT_BREAKER, and EVENT_BREAKER_ENABLE.  The last two are only used by Universal Forwarders, but can be specified anywhere.

I recommend these settings:

sourcetype = job_logs
[source::*\trace*.txt]
TIME_PREFIX = \)
TIME_FORMAT = %d-%m-%y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 18
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TRUNCATE = 10000
EVENT_BREAKER = ([\r\n]+)
EVENT_BREAKER_ENABLE = true 
TRANSFORMS-set= setnull,setparsing
---
If this reply helps you, Karma would be appreciated.
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Karthikeyan,

what are the TIME_PREFIX and TIME_FORMAT for your sourcetype in your props.conf?

They must be:

[your_sourcetype]
TIME_PREFIX = \(\d+\.\d+\)\s+
TIME_FORMAT = %m-%d-%y %H:%M:%S

Ciao.

Giuseppe

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share the props.conf settings for that sourcetype.  One or more of them needs to be adjusted.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...