Hive partitions and timepicker

[pierre_corbel]

Hello,

I got a partitionned Hive table by field dt (in the YYYYMMDD format)

Example :

/mywarehouse/my.db/foo/dt=20160207/part-m-00000

I got a Hunk Index on top of that :

[foo]
vix.provider = my_hive_provider
vix.input.1.path = /mywarehouse/my.db/foo/...
vix.input.1.splitter.hive.dbname = my
vix.input.1.splitter.hive.tablename = foo
vix.input.1.splitter.hive.fileformat = orc

The problem is, when I select a date from the timepicker, I would like Hunk to go directly to the dt partition (because now it makes a full scan of the DB)

I try to modify the following:

vix.input.1.path = /mywarehouse/my.db/foo/${dt}/...

and to add in props.conf :

[foo]
TIME_PREFIX="dt":
TIME_FORMAT = %Y%m%d

[source::.../mywarehouse/my.db/foo/*/*]
sourcetype = foo

But none of it matter...

Could someone lend me a hand on that?

Thanks

[pierre_corbel · Accepted solution]

I finally did like with HDFS Indexes, i.e. add in indexes.conf

vix.input.1.et.format = yyyyMMdd
vix.input.1.et.regex = /mywarehouse/my.db/foo/dt=(\d+)
vix.input.1.lt.format = yyyyMMdd
vix.input.1.lt.offset = 86400
vix.input.1.lt.regex = /mywarehouse/my.db/foo/dt=(\d+)

[rdagan_splunk]

Instead of vix.input.1.path = /mywarehouse/my.db/foo/${dt}/...
try
In the VIX UI, select the option to customize timestamp format
See this document: http://docs.splunk.com/Documentation/Hunk/latest/Hunk/Addavirtualindex