topic Re: generate _time for search in metadata in Splunk Search

How to generate _time for search in metadata?

indeed_2000 — Wed, 23 Nov 2022 14:01:38 GMT

need to generate current date like this "20201123" and use as a search filter on metadata.

AFAIK there is no "_time" in metadata so need to generate current date for search filter.

here is my query,

|metadata type=sources index="app" |table source

any idea?

Thanks,

Re: generate _time for search in metadata

gcusello — Wed, 23 Nov 2022 07:34:10 GMT

Hi @indeed_2000,

if you have events in an index, you must have _time associated to each event, otherwise they weren't indexed!

Anyway, you can use eval and now() to assign the current time value to the _time field:

| metadata type=sources index="app" | eval _time=now() | table _time source

in addition, you can use the addinfo command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Addinfo) to add other information to your search, between them there's the info_search_time that you can use.

Ciao.

Giuseppe

Re: generate _time for search in metadata

bowesmana — Wed, 23 Nov 2022 07:43:31 GMT

metadata command does not always give you what you think - you filter the 3 fields that metadata returns first/last/recent, but I am not sure you will get what you want.

If you are trying to find sources for a particular index within a time window, you are probably better off using tstats, where you can use a _time filter.

Re: generate _time for search in metadata

indeed_2000 — Wed, 23 Nov 2022 08:37:17 GMT

@bowesmana the reason I use metadata is so fast.

i encounter with huge files.

any other idea?

Re: generate _time for search in metadata

yuanliu — Wed, 23 Nov 2022 09:46:43 GMT

This is very confusing. @gcusello already showed how to use now(). Is there still something missing? An example of using now() could be to determine if a source has not updated since today at midnight:

| metadata type=sources index="app" | where recentTime < relative_time(now(), "-0d@d")

Maybe you can explain what is the use of this _time you are trying to generate.

For example, the above use case can also be achieved without where command, as is explained in metadata#Time ranges. Is there something that cannot be done with time picker?

Re: generate _time for search in metadata

indeed_2000 — Wed, 23 Nov 2022 11:40:14 GMT

@yuanliu ok here is the query:

| metadata type=sources index="app"
| eval _time=relative_time (now(), "-1d@d")
| eval time=strftime(_time,"%Y%m%d")
| table source time

here is the result:

/data/app/20221122/CUS/app.log                                                              20221122
/data/app/20221122/CUS/app.log.2022-11-22                               20221122
/data/app/20221119/CUS2/app-exception.log.2022-11-22    20221122
/data/app/20221119/CUS2/app.log.2022-11-22                             20221122

expected result:

/data/app/20221122/CUS/app.log 20221122
/data/app/20221122/CUS/app.log.2022-11-22 20221122

any idea?

Thanks,

Re: generate _time for search in metadata

yuanliu — Thu, 24 Nov 2022 22:31:50 GMT

It is still unclear. If I must speculate, you are concerned about the 3rd path segment in source that resembles a date, and you want to select those that matches yesterday's date. Is this correct? Such intentions may be obvious to you. But none can be certain to anyone else. Not only is the intention absent in text, but also none of your illustrated code contains any selection command.

If the 3rd path segment is of concern, you should first extract that part, then filter based on that field, e.g.,

| metadata type=sources index="app" | rex field=source "/data/app/(?<path_date>\d+)" ``` lots of simplification assumptions here ``` | eval yesterday=strftime(relative_time (now(), "-1d@d"),"%Y%m%d") | where path_date == yesterday