topic Re: Windows: How to upload multiple files with different sourcetypes? in Getting Data In

Windows: How to upload multiple files with different sourcetypes?

edrivera3 — Wed, 01 Apr 2015 19:42:35 GMT

OS: Windows

Hi,

I have a bunch of folders with five files, and I want to index just two of them. These two files have different custom sourcetypes. At this moment, I am uploading one file at a time so this is taking me a lot of time. I would appreciate your help with this matter. Thanks

Re: Windows: How to upload multiple files with different sourcetypes?

aljohnson_splun — Wed, 01 Apr 2015 19:45:49 GMT

Add some details. For example - you only want to index two of what - two of the files, not folders, right? Further more, are all 5 files the same file type? Are the two named in a particular fashion in each folder? Do the folders have a particular structure or naming ?

Re: Windows: How to upload multiple files with different sourcetypes?

tonykung — Wed, 01 Apr 2015 20:04:24 GMT

set up a forwarder then

Re: Windows: How to upload multiple files with different sourcetypes?

masonmorales — Wed, 01 Apr 2015 20:25:21 GMT

Upload them all to a folder on your Splunk server. Then, do:

splunk add oneshot /tmp/yourfolder/file1 -index myindex -sourcetype sourcetypeA
splunk add oneshot /tmp/yourfolder/file2 -index myindex -sourcetype sourcetypeB
splunk add oneshot /tmp/yourfolder/file3 -index myindex -sourcetype sourcetypeC
etc.

Re: Windows: How to upload multiple files with different sourcetypes?

edrivera3 — Wed, 01 Apr 2015 20:35:30 GMT

Ok. I have one directory which contains 70 subdirectories. Each subdirectory have five files with different extensions. I want to upload only two of those five files in each subdirectory. These two files have different sourcetypes and different filename. Also, you can find the same filenames in all 70 subdirectories.

Re: Windows: How to upload multiple files with different sourcetypes?

masonmorales — Wed, 01 Apr 2015 20:38:45 GMT

Assuming you are using Linux, you could produce a bash script using the find command and the splunk oneshot commands I listed below to accomplish what you have described.

Re: Windows: How to upload multiple files with different sourcetypes?

masonmorales — Wed, 01 Apr 2015 20:40:49 GMT

Combine with some bash scripting and voila.

Re: Windows: How to upload multiple files with different sourcetypes?

edrivera3 — Wed, 01 Apr 2015 21:17:53 GMT

No, I'm using Windows but I just found a half solution. Here:
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. In File or Directory, input the directory with recursive (...) i.e C:// blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

I just tried to do the same for the next type of file but splunk doesn't let me select the same directory because it was selected in the previous uploading. Is there a way to make splunk choose the same directory.

Re: Windows: How to upload multiple files with different sourcetypes?

masonmorales — Wed, 01 Apr 2015 21:22:48 GMT

Why not use a lower root of the directory?

Re: Windows: How to upload multiple files with different sourcetypes?

edrivera3 — Wed, 01 Apr 2015 21:31:48 GMT

If I use a lower directory I will be uploading data from other directories that I don't want.

Re: Windows: How to upload multiple files with different sourcetypes?

edrivera3 — Thu, 02 Apr 2015 14:51:26 GMT

Well, I found a solution to my problem.
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. Choose one-time index and select the directory you want to upload. Then add three points after the directory address
for recursive (...) i.e C://blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

After you finish uploading those files, delete the full path to your data in Settings>Data inputs>Files & Directories and repeat the process for the second files which have a different sourcetype.