All Apps and Add-ons

How to configure Hunk for ORC format data in Hive?

toabhishek16
New Member

Hi Team,

I am using Apache Hadoop 2.6.0 and Hive 0.14. I have configured Hunk for text data in hive and HDFS.

but I am not able to configure ORC format data in Hive. I tried properties discussed in (http://answers.splunk.com/answers/210194/settings-to-configure-hunk-for-hive-orc-table.html ) but it is not working. I am getting a garbage values in Hunk.

Please help me by providing settings.

Thanks & Regards
Abhisek

0 Karma

hyan_splunk
Splunk Employee
Splunk Employee

For any hive formats other than text, you need to specify fileformat property. So in your case:

vix.input.1.splitter.hive.fileformat = orc

mikechu
New Member

Hi

I'm using Splunk and Hunk on AWS. We have a Hive external table with textfile format. It's field delimiter is comma. I'm able to setup a index provider and virtual index for the hive table with the following setup. However, the field values are not parsed properly. It seems Splunk put the entire row to the 1st field. Is there a vix.input.1.hive.??? parameter for hive field delimiter?

vix.input.1.splitter.hive.columnnames=id,source,type,amount,status,from_pin,region_id,ip_address,date_closing,date_created,date_event,user_seller,account_seller,advertiser_id,user_id,account_id,sale_type,sale_item,sale_id,sale_amount,request,order_id,from_co,client_id,product_price,utcdate,bonus
vix.input.1.splitter.hive.columntypes=string:string:string:double:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string
vix.input.1.splitter.hive.dbname=rs_analytics
vix.input.1.splitter.hive.tablename=transaction
vix.input.1.splitter.hive.fileformat=textfile

0 Karma

hyan_splunk
Splunk Employee
Splunk Employee

Yes.
vix.input.1.splitter.hive.rowformat.fields.terminated=,

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Also, it seems like the provider may be missing this flag
vix.hive.metastore.uris = thrift://metastore.example.com:9083

0 Karma

toabhishek16
New Member

Hi please find below current settings:

Provider:
vix.splunk.search.splitter=HiveSplitGenerator
and all other properties are set to default

Virtual Index:
Path to data in HDFS =
vix.input.1.splitter.hive.dbname = default
vix.input.1.splitter.hive.tablename = hivetable_orc

the above setting is working fine with text files but not working with ORC format.

please help...

0 Karma

apatil_splunk
Splunk Employee
Splunk Employee

Can you please post how your existing settings look like (provider + virtual index).

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...