How to access Date Partitioned files in HDFS dynamically using virtual index

[sdaruna]

Hi,

I have hdfs folders as below.

/bla/bla/bla/20160121
/bla/bla/bla/20160122
/bla/bla/bla/20160123

How to access the data matched in any specific date only for a given query.? lets say, i would like to get data from 20160121 folder only. I do not want to create one virtual index for each folder, coz we gonna have date for all 365 days in a year.

[jbjerke_splunk]

Hi sdaruna

You should define your virtual index to include subfolders recursively and define the time as part of the file source path. You can then control the data returned in Splunk searches by using the Splunk time picker (2016-01-21)

Path to files in HDFS:
/bla/bla/bla

Time capturing regex:
/bla/bla/bla/\d+

Time format:
yyMMdd

The docs for this is here:
http://docs.splunk.com/Documentation/Hunk/latest/Hunk/Addavirtualindex

If this is the answer you were looking for, please mark it as Answered.

j