Temporal Lookups

[craigmunro]

Hi, I was hoping to use a lookup table to add some fields but it doesn't seem to do quite what I was hoping.

I have the following file, lookup.csv:

timestamp,host,env
2010-02-28 17:26:00,host1,dev
2010-02-28 16:50:00,host2,uat
2010-02-28 16:34:00,host1,uat

and I would like events from host1 to have the field env=uat for events up until 17:26 and env=dev for events after that. Instead all events have the field env=dev.

My transforms.conf looks like:

[mylookup]
filename = lookup.csv
time_field = timestamp
time_format = %Y-%m-%d %H:%M:%S
default_match = unknown
case_sensitive_match = false

Is this expected? Is there another way to achieve what I'm looking for?

Thanks

[Stephen_Sorkin · Accepted solution]

The stanza above should be in transforms.conf, since it defines the lookup table. To make it apply for a given sourcetype, you should add to props.conf:

[<sourcetype>]
LOOKUP-env = mylookup host OUTPUT env

Editing to include new findings:

1. I can't reproduce the problem using time_format and rendered time, provided I change it to 2011, not 2010 (I haven't tried with 2010).
2. It works if I use UTC epoch time, like 1298942760,host1,dev and take out the time_format attribute in transforms.conf.

Are you sure that you're looking at data from 2010 and not 2011?

[craigmunro]

Yes, that did it. Changing to the correct year (!) and using UTC time everything now works.

Thanks!

[craigmunro]

Thanks Stephen, actually the stanza is in transforms.conf and I have the correct entry in props.conf I've corrected my question.