Splunk Search

How to generate _time for search in metadata?

indeed_2000
Motivator

Hi

need to generate current date like this "20201123" and use as a search filter on metadata.

AFAIK there is no "_time" in metadata so need to generate current date for search filter.

 

here is my query, 

|metadata type=sources index="app" |table source

 

any idea?

Thanks,

 

Labels (4)
Tags (4)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

It is still unclear.  If I must speculate, you are concerned about the 3rd path segment in source that resembles a date, and you want to select those that matches yesterday's date.  Is this correct?  Such intentions may be obvious to you.  But none can be certain to anyone else.  Not only is the intention absent in text, but also none of your illustrated code contains any selection command.

If the 3rd path segment is of concern, you should first extract that part, then filter based on that field, e.g.,

| metadata type=sources index="app"
| rex field=source "/data/app/(?<path_date>\d+)" ``` lots of simplification assumptions here ```
| eval yesterday=strftime(relative_time (now(), "-1d@d"),"%Y%m%d")
| where path_date == yesterday

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

metadata command does not always give you what you think - you filter the 3 fields that metadata returns first/last/recent, but I am not sure you will get what you want.

If you are trying to find sources for a particular index within a time window, you are probably better off using tstats, where you can use a _time filter.

 

 

indeed_2000
Motivator

@bowesmana the reason I use metadata is so fast.

i encounter with huge files.

any other idea?

0 Karma

yuanliu
SplunkTrust
SplunkTrust

This is very confusing.  @gcusello already showed how to use now().  Is there still something missing?  An example of using now() could be to determine if a source has not updated since today at midnight:

| metadata type=sources index="app"
| where recentTime < relative_time(now(), "-0d@d")

Maybe you can explain what is the use of this _time you are trying to generate.

For example, the above use case can also be achieved without where command, as is explained in metadata#Time ranges.  Is there something that cannot be done with time picker?

indeed_2000
Motivator

@yuanliu ok here is the query:

| metadata type=sources index="app"
| eval _time=relative_time (now(), "-1d@d")
| eval time=strftime(_time,"%Y%m%d")
| table source time

here is the result:

/data/app/20221122/CUS/app.log                                                                20221122
/data/app/20221122/CUS/app.log.2022-11-22                                 20221122
/data/app/20221119/CUS2/app-exception.log.2022-11-22    20221122
/data/app/20221119/CUS2/app.log.2022-11-22                             20221122

 

expected result:

/data/app/20221122/CUS/app.log                                                                20221122
/data/app/20221122/CUS/app.log.2022-11-22                                 20221122

 

any idea?

Thanks,

0 Karma

yuanliu
SplunkTrust
SplunkTrust

It is still unclear.  If I must speculate, you are concerned about the 3rd path segment in source that resembles a date, and you want to select those that matches yesterday's date.  Is this correct?  Such intentions may be obvious to you.  But none can be certain to anyone else.  Not only is the intention absent in text, but also none of your illustrated code contains any selection command.

If the 3rd path segment is of concern, you should first extract that part, then filter based on that field, e.g.,

| metadata type=sources index="app"
| rex field=source "/data/app/(?<path_date>\d+)" ``` lots of simplification assumptions here ```
| eval yesterday=strftime(relative_time (now(), "-1d@d"),"%Y%m%d")
| where path_date == yesterday

 

gcusello
SplunkTrust
SplunkTrust

Hi @indeed_2000,

if you have events in an index, you must have _time associated to each event, otherwise they weren't indexed!

Anyway, you can use eval and now() to assign the current time value to the _time field:

| metadata type=sources index="app" 
| eval _time=now()
| table _time source

in addition, you can use the addinfo command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Addinfo) to add other information to your search, between them there's the info_search_time that you can use.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...