Unable to connect to mongoDB with Hunk through Hunk App for MongoDB

[sharan4u]

I am trying to connect Hunk with mongoDB to visualize mongoDB data in Hunk.
I have installed Hunk 6.2.1 and also installed Hunk_App_for_MongoDB_101 on top of that.

I have set the Virtual Index to connect to the mongoDB instance running on the remote server.

The error displayed in search log is :

ERROR ScriptRunner - Couldn't start child process. script="/usr/java/jdk1.7.0_55-cloudera -Xmx512m -classpath /opt/hunk/bin/jars/SplunkMR-s6.0-h1.0.jar:/opt/hunk/etc/apps/MongoDBApp/bin/hunk_mongodb_app.jar:/opt/hunk/etc/apps/MongoDBApp/bin/lib/* com.splunk.erp.mongodb.MongoDBERP"

Virtual index(index.conf) snapshot is as follows :

[provider:local-mongodb]
vix.mongodb.host = 10.47.33.35:27017
vix.output.buckets.max.network.bandwidth = 0
vix.mongodb.auth.mechanism = CR
vix.mongodb.auth.password = test1
vix.mongodb.auth.username = test
vix.hadoop.security.authorization = 0
vix.command = /usr/java/jdk1.6.0_31

[mongodb_vix]
vix.input.1.path = /root/Downloads/
vix.mongodb.collection = local
vix.mongodb.db = startup_logs\

I am not sure what value we need to specify for "HDFS path(vix.input.1.path)"..Why is it asking "HDFS path" while creating virtual index for connecting mongoDB?

Also let me know if it is necessary to provide Java Home path (vix.command ), if yes, then if we need to provide java home path of the server where Hunk is running or the Java home path of the server where mongoDB instance is running. Kindly suggest.

If anyone has come across this kind of issue, kindly suggest how to debug it.

[naisanza]

This was the solution that worked for me (with an error, return_code=1)

https://answers.splunk.com/answers/316864/hunk-app-for-mongodb-why-cant-i-connect-to-mongodb.html

To reiterate:

Edit $SPLUNK_HOME/etc/apps/MongoDB/default -> indexes.conf
Replace: SplunkMR-s6.0-h1.0.jar
With: SplunkMR-h1.jar

SplunkMR-s6.0-h1.0.jar does not exist. Though it gets loaded and it just gladly accepts the parameters without any checking

04-01-2016 02:58:49.769 INFO  ERP.local-mongodb - Starting: /opt/java/jre1.8.0_77/bin/java -Xmx512m "-classpath" "/opt/splunk/bin/jars/SplunkMR-s6.0-h1.0.jar:/opt/splunk/etc/apps/MongoDBApp/bin/hunk_mongodb_app.jar:/opt/splunk/etc/apps/MongoDBApp/bin/lib/*" "com.splunk.erp.mongodb.MongoDBERP"

[rdagan_splunk]

Try this:
1) Go to hunk/etc/apps/search/local
2) You should be able to see the indexes.conf.

Add your VIX stanza. For example,

[mongodb_movie_rating]
vix.mongodb.collection = movieratings
vix.mongodb.db = raanan
vix.mongodb.field.time = ts
vix.mongodb.field.time.format = date
vix.provider = local-mongodb

[mongodb_movetest]
vix.mongodb.collection = testmoviejson
vix.mongodb.db = raanan
vix.mongodb.field.time = date
vix.mongodb.field.time.format = epoch
vix.provider = local-mongodb

3) Restart Hunk hunk/bin/splunk restart

[sharan4u]

@rdagan_splunk :which version of hunk are you using...Is it asking you "HDFS path" while creating virtual index in Hunk to connect to MongoDB ?
Are you connecting to mongoDB instance running in the local server or remote server. ?

[rdagan_splunk]

I am using the latest version of Hunk and MongoDB App. I also see that if you try to create the VIX from within the UI it is asking you for HDFS path. Therefore, you should create it in the indexes.conf file and restart Hunk (the above examples work for me).
I tried both local and remote connections to MongoDB.

[sharan4u]

Ok..Thank you rdagan_splunk..I will try doing the same and will post here if i face any further issues.

[sharan4u]

Hi,
I have configured indexes.conf file from hunk/etc/apps/MongodbApps/local path as follows. But still am unable to connect to the mongoDB instance running on local server.
[provider:local-mongodb]
vix.command = /usr/java/jdk1.7.0_55-cloudera/bin/java
vix.mongodb.auth.mechanism = CR
vix.mongodb.auth.password = test1
vix.mongodb.auth.username = test
vix.output.buckets.max.network.bandwidth = 0
vix.mongodb.host = 10.47.33.35:27017

[mongodb_vix]
vix.mongodb.collection = sample1
vix.mongodb.db = admin
vix.mongodb.field.time = _id
vix.mongodb.field.time.format = ObjectId
vix.provider = local-mongodb

Search.log displays follwing error :
ERROR ERP.local-mongodb - com.mongodb.MongoException: not authorized for query on admin.sample1
ERROR ERP.local-mongodb - at com.mongodb.QueryResultIterator.throwOnQueryFailure(QueryResultIterator.java:214)
ERROR ERP.local-mongodb - at com.mongodb.QueryResultIterator.init(QueryResultIterator.java:198)
ERROR ERP.local-mongodb - at com.mongodb.QueryResultIterator.initFromQueryResponse(QueryResultIterator.java:176
ERROR ERP.local-mongodb - at com.mongodb.QueryResultIterator.(QueryResultIterator.java:64) ERROR ERP.local-mongodb - at com.splunk.erp.mongodb.MongoDBERP.readEventsFromDB(Unknown Source) ERROR ERP.local-mongodb - at com.splunk.erp.mongodb.MongoDBERP.getEvents(Unknown Source) ERROR ERP.local-mongodb - at com.splunk.erp.mongodb.MongoDBERP.main(Unknown Source)

[rdagan_splunk]

Based on the search log it looks like a security issue . Try to remove (or modify) these flags - then restart:
vix.mongodb.auth.mechanism = CR
vix.mongodb.auth.password = test1
vix.mongodb.auth.username = test
vix.output.buckets.max.network.bandwidth = 0

[sharan4u]

I have removed the following lines from indexes.conf file..Still facing the same error..
vix.mongodb.auth.mechanism = CR
vix.mongodb.auth.password = test1
vix.mongodb.auth.username = test
vix.output.buckets.max.network.bandwidth = 0

[rdagan_splunk]

If all you see in the search.log is com.mongodb.MongoException: not authorized for query on admin.sample1
that means that you can connect to MongoDB, but unable to access that DB / Collection as user test / password test1

On your Mongodb instance (10.47.33.35:27017) with user test / password test1:
1) Can you type ‘ show dbs ‘?
2) Then type ‘ use admin ‘ ?
3) ‘ show collections ‘ ?
4) And finally can you type ‘ db.sample1.find() ‘ ?

[sharan4u]

@rdagan : Yes I can perform all the operations on my Mongodb instance (10.47.33.35:27017) with user test / password test1

[sharan4u]

I am trying to connect to MongoDB replication server...Does the connection from hunk to MongoDB works only for Standalone Instance ?

[rdagan_splunk]

You you are correct. This replication feature was not tested.

[sharan4u]

@rdagan : I am able to connect to MongoDB instance running in Standalone mode. Thanks for your help. Let me know if you are testing on connecting to replication cluster.

[rdagan_splunk]

Great to hear it is working.

And if we decide to upgrade the App to include support for replication cluster I will update this thread.

[sharan4u]

Thank you..

[sharan4u]

@rdagan_splunk : I am not able to see "local" directory in hunk/etc/apps/search...

[rdagan_splunk]

See if you can find the indexes.conf under hunk/etc/apps/Mongodb/local

[Ledion_Bitincka · Accepted solution]

You don't need to specify a vix.input.1.path and it's shouldn't be asking for it (I'll create an bug report for that). Also in vix.command you need to specify the full path to java executable, eg vix.command = /usr/java/jdk1.6.0_31/bin/java - below is a working config from my system

[provider:local-mongodb]
vix.command = /usr/java/latest/bin/java
vix.family              = mongodb_erp_family
vix.mongodb.host        = localhost:27017

[mongodb_vix]
vix.provider           = local-mongodb
vix.mongodb.db         = hunk
vix.mongodb.collection = test
vix.mongodb.field.time = _id
vix.mongodb.field.time.format = ObjectId

[sharan4u]

Have you connected to mongoDB instance running on same server where hunk is running..or you have connected to the remote mongoDB server ?

I am trying to connect to mongoDB instance running in remote server. Hunk is failing to connect.
Its throwing below error :
[local-mongodb] Error while running external process, return_code=1. See search.log for more info

In search.log am not finding any error. Let me know if any workaround is required.

[naisanza]

My MongoDB service is running remotely, and it is accessible

root@hunk-running:/opt/mongodb/mongodb-linux-x86_64-ubuntu1410-clang-3.2.4/bin# ./mongo 10.0.2.69
MongoDB shell version: 3.2.4
connecting to: 10.0.2.69/test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
> show dbs
admin   (empty)
local   0.078GB
portal  7.950GB
>