Getting Data In

Windows: How to upload multiple files with different sourcetypes?

Builder

OS: Windows

Hi,

I have a bunch of folders with five files, and I want to index just two of them. These two files have different custom sourcetypes. At this moment, I am uploading one file at a time so this is taking me a lot of time. I would appreciate your help with this matter. Thanks

0 Karma
1 Solution

Builder

Well, I found a solution to my problem.
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. Choose one-time index and select the directory you want to upload. Then add three points after the directory address
for recursive (...) i.e C://blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

After you finish uploading those files, delete the full path to your data in Settings>Data inputs>Files & Directories and repeat the process for the second files which have a different sourcetype.

View solution in original post

Builder

Well, I found a solution to my problem.
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. Choose one-time index and select the directory you want to upload. Then add three points after the directory address
for recursive (...) i.e C://blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

After you finish uploading those files, delete the full path to your data in Settings>Data inputs>Files & Directories and repeat the process for the second files which have a different sourcetype.

View solution in original post

Influencer

Upload them all to a folder on your Splunk server. Then, do:

splunk add oneshot /tmp/yourfolder/file1 -index myindex -sourcetype sourcetypeA
splunk add oneshot /tmp/yourfolder/file2 -index myindex -sourcetype sourcetypeB
splunk add oneshot /tmp/yourfolder/file3 -index myindex -sourcetype sourcetypeC
etc.

Influencer

Combine with some bash scripting and voila.

0 Karma

Splunk Employee
Splunk Employee

Add some details. For example - you only want to index two of what - two of the files, not folders, right? Further more, are all 5 files the same file type? Are the two named in a particular fashion in each folder? Do the folders have a particular structure or naming ?

0 Karma

Builder

Ok. I have one directory which contains 70 subdirectories. Each subdirectory have five files with different extensions. I want to upload only two of those five files in each subdirectory. These two files have different sourcetypes and different filename. Also, you can find the same filenames in all 70 subdirectories.

0 Karma

Influencer

Assuming you are using Linux, you could produce a bash script using the find command and the splunk oneshot commands I listed below to accomplish what you have described.

0 Karma

Builder

No, I'm using Windows but I just found a half solution. Here:
1. In Splunk just go to Settings>Data inputs>Files & Directories.
2. Select New
3. In File or Directory, input the directory with recursive (...) i.e C:// blabla/blabla/...
4. In whitelist, input the extension i.e. (.stat$)
5. Finally, choose one sourcetype for all the data and it is done.

I just tried to do the same for the next type of file but splunk doesn't let me select the same directory because it was selected in the previous uploading. Is there a way to make splunk choose the same directory.

0 Karma

Influencer

Why not use a lower root of the directory?

0 Karma

Builder

If I use a lower directory I will be uploading data from other directories that I don't want.

0 Karma

New Member

set up a forwarder then

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!