Splunk Search

How to make another field date and time field before and after indexing

vrmandadi
Builder

Hello Splunkers,

I have data where the index time is different from the actual file.The source has the correct date and time.I want  splunk to use it as index time and if that is not possible I want to extract date and time from the source field ,create new field and use that as date and time field.Below is the example of the source file.

 

Source:

/admin/logs/abc/inventory/04-04-2022-101634-all-b5.xxx

 

1. I want splunk to take 04-04-2022-101634 and use it as indexed time like  04-04-2022 -10:16:34(dd-mm-yyyy-hh:mm:ss) . I want the props.conf 

2. Also if the data is indexed without using the source file .I want to extract teh date and time from the source and create and new field called correct_time as use as _time

 

 

Thanks in Advance

Labels (1)
0 Karma
1 Solution

mayurr98
Super Champion

Sorry you mentioned dd-mm-yyyy-hh:mm:ss this format.

if your format is mm-dd-yyyy-hhmmss below should work for all the sources.

| makeresults 
| eval source="/admin/logs/abc/inventory/04-04-2022-101634-all-b5.xxx" 
| eval _time=strptime(replace(source,".*(?=/)/",""),"%m-%d-%Y-%H%M%S")

 

View solution in original post

mayurr98
Super Champion

Example

File Name:/admin/logs/abc/inventory/04-04-2022-101634-all-b5.xxx
File Name Format: %d-%m-%Y-%H%M%S-all-b5.xxx
_time value assigned to events:2022-04-04 10:16:34

props.conf

[mysourcetype]
TRANSFORMS=timestampeval

transforms.conf

[timestampeval]
INGEST_EVAL = _time=strptime(replace(source,".*(?=/)/",""),"%d-%m-%Y-%H%M%S")

This takes the "source" metadata value (which is the path and file name), removes the path, then extracts the date from the filename. The time defaults to 00:00:00.

All events in the file will have the same _time when imported.

 

Run below anywhere search to test 

| makeresults 
| eval source="/admin/logs/abc/inventory/04-04-2022-101634-all-b5.xxx" 
| eval _time=strptime(replace(source,".*(?=/)/",""),"%d-%m-%Y-%H%M%S")
0 Karma

vrmandadi
Builder

@mayurr98 Thank you very much for your reply .  it did not work on the below source.

 

/admin/logs/su0051/inventory/02-22-2022-070801-all-v4.fru

/admin/logs/laaaa0016/inventory/11-16-2021-145102-all.fru

0 Karma

mayurr98
Super Champion

Sorry you mentioned dd-mm-yyyy-hh:mm:ss this format.

if your format is mm-dd-yyyy-hhmmss below should work for all the sources.

| makeresults 
| eval source="/admin/logs/abc/inventory/04-04-2022-101634-all-b5.xxx" 
| eval _time=strptime(replace(source,".*(?=/)/",""),"%m-%d-%Y-%H%M%S")

 

vrmandadi
Builder

Thank you that worked

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You should use ingest-time eval. See the https://conf.splunk.com/files/2020/slides/PLA1154C.pdf There is an example about correcting _time.

The correct_time extraction/calculation could be done in search time like any other field.

0 Karma

vrmandadi
Builder

Thank you for your reply.But I am not sure how to do the eval on it.Can you please help me on it and also the regex part...I can do regex but I am not sure how to add " :" for the time

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...