Archive
Highlighted

Dump command splits results in many files. How to consolidate to have a single file instead of splitting results by source?

Explorer

I have used the dump command to extract data from production server and play with it on my local.
I have 6 different hosts in Production, so I'd like to run 6 dumps so that the host segregation is respected.
I used something like this:

sourcetype=MySourceType host = node1 | dump basefilename=MySourceType-node1-2016Jan01ToMay31 

It worked, but the results were split by 'source', as in the source file name.
The log files where the data comes from are located in something like

\\node1.company.com\logs\logfile1.log

There are 32 logfiles , so my dump results end up like this:

$SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/node1/node1.company.com/logs/logfile1.log/MySourceType-node1-2016Jan01ToMay31_450_10.raw.gz
$SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/node1/node1.company.com/logs/logfile1.log/MySourceType-node1-2016Jan01ToMay31_816_40.raw.gz
$SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/node1/node1.company.com/logs/logfile2.log/MySourceType-node1-2016Jan01ToMay31_450_21.raw.gz
$SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/node1/node1.company.com/logs/logfile2.log/MySourceType-node1-2016Jan01ToMay31_816_51.raw.gz

I think the 450 and 816 might be me trying to run the dump once, cancelling and doing it again with a different time frame.
My main issue is, how can I consolidate the dump to have a single file, instead of having it segregated by source?

Tags (2)
0 Karma
Highlighted

Re: Dump command splits results in many files. How to consolidate to have a single file instead of splitting results by source?

SplunkTrust
SplunkTrust

Hi icegras, one possible answer might be to eval source to some common value for all events in question, i.e.

 sourcetype=MySourceType host = node1 | eval source="mycommonsource" | dump basefilename=MySourceType-node1-2016Jan01ToMay31

Please let me know if this works out for you! 😄

View solution in original post

Highlighted

Re: Dump command splits results in many files. How to consolidate to have a single file instead of splitting results by source?

Explorer

I did not think I could overwrite a default field like source, but that did the trick!
Still split them up in 9 files, but that's because the default max size for the gz file is 64mb. You want to keep that because once unpacked it's about 250mb each, and max size to upload a file in order to add data is 500mb.

0 Karma