How do I configure et/lt offset for leap year or months with different days 28/29/30/31 days?

[melonman]

Hi,

I want to know how to configure offset for et/lt in indexes.conf for virtual index in Hunk.

I understand the basic configuration where HDFS path contains the following date information.

/data/test/20140213/07/some.log.gz
/data/test/20140213/08/some.log.gz
/data/test/20140213/09/some.log.gz

vix.input.1.et.format = yyyyMMddHH
vix.input.1.et.offset = 0
vix.input.1.et.regex = test/(\d+)/(\d+)/
vix.input.1.lt.format = yyyyMMddHH
vix.input.1.lt.offset = 3600
vix.input.1.lt.regex = test/(\d+)/(\d+)/
vix.input.1.path = /data/test/${yyyymmdd}/${hh}/...

It is clear that the offset is 1 hour = 3600

However, I don't know how to configure this:

/data/test/2014/some.log.gz
/data/test/2013/some.log.gz
/data/test/2012/some.log.gz

vix.input.1.et.format = yyyy
vix.input.1.et.offset = 0
vix.input.1.et.regex = test/(\d+)/
vix.input.1.lt.format = yyyy
vix.input.1.lt.offset = <ummmm I don't know>
vix.input.1.lt.regex = test/(\d+)/(\d+)/
vix.input.1.path = /data/test/${yyyymmdd}/${hh}/...

If the path has only year information, then do I need to consider leap year?
If the path has year and month, do I also have to consider number of days in a month? 28/29/30/31??

Thanks,

[Splunk_Shinobi · Accepted solution]

For variable partitions like month/year level,
you should set the offsets such that they include the maximum possible time range

e.g.

month = 31 days = 2678400 seconds
year = 366 days = 31622400 seconds