Getting Data In

How do I forward log files to Hunk?

gibu_george
Engager

Hi All,

I am new to Hunk and would like to know if it is possible to directly send log file data to Hunk using the Splunk forwarder?
What is the recommended way to do this? Could you also share a link to docs for the same?

Thanks in advance.
--gibu

0 Karma
1 Solution

ChrisG
Splunk Employee
Splunk Employee

Are you talking about forwarding typical log data to a regular Splunk index? If so, Hunk works exactly the way Splunk Enterprise does, and you can use the Splunk Universal Forwarder documentation.

View solution in original post

kschon_splunk
Splunk Employee
Splunk Employee

By "forwarding to Hunk" I assume you mean placing the data on HDFS (or another Hadoop-compatible file system) to be searched via a virtual index?

The most common workflow is to add the log files to a regular Splunk index (using any input method, such as a Splunk forwarder), and set up that index to archive to HDFS. This means you can use a Splunk-managed index to get fast performance on the most recent data, and you can use Hadoop (via Hunk) to search the much larger pool of older data. You can find more information about getting data into Splunk indexes here:

http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Getstartedwithgettingdatain

You can find documentation about archiving here:

https://docs.splunk.com/Documentation/Hunk/6.4.3/Hunk/ArchivingSplunkindexes

If you want to copy the data directly to HDFS without first adding it to a regular Splunk index, you cannot currently do this via a Splunk forwarder. There are a number of third party tools that can be used to do this, e.g. Apache Flume (https://flume.apache.org/).

ChrisG
Splunk Employee
Splunk Employee

Are you talking about forwarding typical log data to a regular Splunk index? If so, Hunk works exactly the way Splunk Enterprise does, and you can use the Splunk Universal Forwarder documentation.

View solution in original post

.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!