The error message contains the directory and its permissions of where Splunk is failing to write. A simple fix here would be to have Splunk run as user hdpuser instead of splunkd1 Permission denied: user=splunkd1, access=WRITE, inode="/tmp/hadoop-yarn/staging/splunkd1/.staging":hdpuser:supergroup:drwxr-xr-x ... View more

I assume your don't see any events if you run a search over a given time period (hour, day, week etc) - correct? If so this is generally an indication that the time extracted from the events is different from the one extracted from the path. Is there a timezone difference between the path and Splunk's or your user's setting in Splunk? I highly recommend that you do specify the timezone of the extracted time from the path vix.input.[N].et/lt.timezone - the timezone in which to interpret the extracted time. E.g. "America/Los_Angeles" or "GMT-8:00", see http://docs.oracle.com/javase/7/docs/api/java/util/TimeZone.html for more info on the supported strings ... View more

Are the events time stamped correctly? ie does the event timestamp correspond to the path of the file where the event is found? Unrelated, I'd recommend using the following more efficient search to get a table of computerName (and a count of events) index=fs_vindex | stats count BY computerName ... View more

Quick clarification ERP stands for "external results provider" - in Splunk the provider of results/events are the native indexes, in Hunk the results/events are provided by an external source, Hadoop, MongoDb, etc ... View more

Yes there is, you just have to: a) have the appropriate hadoop libs available on the Hunk node and b) set HADOOP_HOME differently for the different providers belonging to the different versions of the clusters ... View more

No, just 1, we automatically recognize field=value pairs in paths and you can look at the docs here for how to configure Hunk to recognize timestamps. Also you can look at this blog post for a bit more info ... View more

All great comments, I just want to highlight that Hunk is not optimized for search (for many reasons) but rather analytics workloads - in many cases the analytics performance could be comparable to that of Splunk. That said, if your uses case is primarily search, why not send the data to Splunk indexers? There's also ways that data can be organized and laid out, hierarchically to further optimize Hunk performance - e.g. /some/base/path/host=abc/20150910/... ... View more

What does your full search look like? transaction requires that the events it sees be in descending time order and it should throw an error stating such - unless of course you force ordering of the events using sort as one of the commands before transaction ... View more

A couple of questions: Are you getting any errors in the UI or search.log? Also, what search are you trying to run? What version of Hunk are you using? Are you using EMR + Hunk hourly? (I'm a bit puzzled as I tried this with vix.input.1/2.path pointing to buckets in different regions and searches worked ok) ... View more

What happens if the vix points only to the east bucket? Also are you able to access the east bucket using the hadoop cli - hadoop fs -ls s3n://bucket-us-east1-elb-logs/ ? ... View more

Since you're trying to access buckets from different regions can you try the following: At the provider level, set [provider:abc] ... vix.fs.s3n.endpoint = s3.amazonaws.com [elb] vix.provider = abc vix.input.1.path = s3n://bucket-us-west2-elb-logs/... vix.input.2.path = s3n://bucket-us-east1-elb-logs/... ... Note that accessing buckets cross region has performance and cost implications ... View more

Also, make sure to set the same timezone in indexes.conf for vix.input.1.et/lt.timezone - in many cases the timezone of the data is GMT while the search is ran in a user specified timezone, e.g. PST ... View more

Usually the headless hdfs kerberos principal does is a Service Principal and does not have access to the compute resources of Hadoop. Have you tried using a "User Principal" vix.kerberos.keytab = /etc/security/keytabs/hdfs.headless.keytab vix.kerberos.principal = hdfs@<REALM> ... View more

To connect to indexers, the same Splunk Enterprise ports are needed (8089 by default). For Hadoop, ports depend on the Hadoop distro - indexers need access to the Secondary/Namenode (usually port 8020), and DataNode transfer ports (usually port 50010). Indexer Splunk version doesn't matter, but you need Java and Hadoop libraries on the indexers. ... View more

Do all indexer machines need access to hadoop (hdfs)? Yes, they need network access, Java and Hadoop libraries (ie you should be able to successfully run hadoop fs -copyFromLocal .... from every indexer) Does hunk copy the files locally and than to hdfs? No, the buckets are copied from indexers directly to HDFS ... View more

The Google Maps Add-on (https://splunkbase.splunk.com/app/368/) would be a good starting place, it should be fully compatible with Hunk even though the name doesn't mention it ... View more

Hunk EMR should automatically detect any new cluster, within a few minutes of the cluster being ready. There's one caveat though, that the cluster needs to be in the same region as the Hunk node. ... View more

Regex and stanza are shown correctly (ie no format messup) ... View more

Are you using Splunk or Hunk? If you're using Splunk you need to place the configs (in your original question) on an indexer. What you've commented above does search time field extractions and lives in the search head and should work in Splunk and Hunk. ... View more

Currently Splunk does not provide a way to write data directly to HDFS. Splunk forwarders can currently send data to either a Splunk indexer or a syslog receiver, however we're planning to expand that list. If you don't mind commenting what type of data are you trying to collect and what functionality would you want in the "collector" (parse, partition, filter, etc) ? ... View more

Yes, index=[star] would match any index that the user (in this case system) is allowed to search - except internal indexes (the ones that start with _, like _internal) ... View more

The "Application does not exist: splunk_app_microsoft_exchange" error message should be a temporary error message until the configuration bundles stabilize in Hunk. Did this error go away within a few minutes or does it persist? ... View more