Splunk Search

Overriding _time through a calculated field

bowesmana
SplunkTrust
SplunkTrust

I have created a calculated field which parses _time from a date stamp in the data.

However, it does not set _time correctly. If I set the calculated field to something different it's fine.

So, was just wondering if there was any documentation anywhere that talks about being able to override _time with a calculated field.

NB: I can't set the event _time at ingestion to be the correct date from the data as I am ingesting a complete data set every day, where historical results may change, so I'm just using a 24h search and then changing _time.

 

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

First things first - this is my understanding of how splunk seems to work but it can be completely wrong 😉 so do your testing.

Having said that - since you can freely (re)define _time by evals in your search I suppose there shouldn't be much of a problem to do a calculated field.

But remember that calculated fields are applied after searching. So you wouldn't be able to limit the timerange normally. If you - let's say - ingested a batch of events on every midnight, all those events would go to the bucket associated with that point in time and would be searchable if the time range included that midnight. You might later recalculate the _time field and use it for stats, for example, but still the actual data on indexers would exist only on midnight.

The only solution here would be to adjust the time at ingest time (why not use ingest-time evals?)

EDIT: I did some tests out of curiousity and it seems you can create an eval for _time but the search-head doesn't seem to take it into account. So apparently you can easily overwrite _time in your search but as a calculated field it doesn't work. At least for me.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

That was my point, that a calculated field of _time=strptime(...) does not provide _time with the calculated value, but that same eval statement as part of the search pipeline does.

I was looking to see if there was any Splunk official doc that explained why this was the case.

I can't calculate time at ingest for the reason mentioned, i.e. I would have to do an all time search every time I want the data, which would get every single event for all daily ingests, and then have to filter on indextime to remove the ones that I don't want.

 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...