Splunk Search

Dump Command - How to Save Files on Remote Path and Question About maxlocal

steverimar
Explorer

We're trying to export data out of a very large splunk index using the dump command into multiple csv files where the min file size is 1GB of each file.

E.g here's our command ...|dump basefilename=netezza rollsize=1024 format=csv

What we want is to have the files saved on a remote location not local on the Splunk server. Is there a way to do this?

Also, what does the maxlocal parameter mean? We're a little confused by that. It says once it reaches 1GB it moves it to HDFS? We don't have HDFS configured, what happens once it reaches 1GB? Is there a way to make maxlocal unlimited?

Tags (1)

livehybrid
Builder

Did you get this resolved?
I've done something similar, there are a couple of ways to get the data onto a remote server.
In the past I have setup a SSHFS mount at $SPLUNK_HOME/var/run/splunk/dispatch, note that this location will be used for all searches.
Stop Splunk before mounting the directory, you'll need to temporarily move out the old directory in order to mount it correctly. Check the file permissions are correct and then start Splunk.
The output of your DUMP commands will now be in $SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/ where is the ID of your search.
It isnt the most elegant solution but it worked for me.

Hope this helps!

0 Karma

lstewart_splunk
Splunk Employee
Splunk Employee

The maxlocal argument never worked properly and has been removed from the code and the documentation.

0 Karma

kausar
Path Finder

@steverimar were you able to figure this out?
I am using Splunk 6.4.2. With dump command, when I provide maxlocal argument, it complains and returns error 'invalid argument' (not using HDFS). Did you find a way to save the output in remote system?

millern4
Communicator

Did you ever come across a way to do this or get it to work correctly.

I could see this being useful in an incident response investigation where we needed to give a large data set out to vendor or 3rd party for analysis rather than searching within splunk.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...