Getting Data In

Upload a file - source?

gelica
Communicator

Hi Splunkers!

I have a question regarding indexing new data.

I'm using the file path to extract some of my fields, like id and date.
My paths looks something like this:

dir/555488/dir_2013-07-26_09-08-00/file

where the 555488 is the id and 2013-07-26_09-08-00 is the date I'm extracting.

This works fine when I'm using monitors to index the files, but if I want to upload just one file using splunk's "Upload and index a file"-option, the source won't be the whole path, just the file name.

It isn't possible for me to monitor all my data, so I wonder if there is a way around this issue?

0 Karma
1 Solution

sowings
Splunk Employee
Splunk Employee

I don't know how to do this in the UI, but if you use the command line tool "splunk add oneshot" you can use the -source argument to specify the full path to the file, and it will be carried over into the "source" metadata field. More data can be found here:

http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/MonitorfilesanddirectoriesusingtheCLI

View solution in original post

0 Karma

sowings
Splunk Employee
Splunk Employee

I don't know how to do this in the UI, but if you use the command line tool "splunk add oneshot" you can use the -source argument to specify the full path to the file, and it will be carried over into the "source" metadata field. More data can be found here:

http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/MonitorfilesanddirectoriesusingtheCLI

View solution in original post

0 Karma

gelica
Communicator

Thank you! I've read that add oneshot and spool did the same thing, so I only tried spool, which didn't use my source.

0 Karma

Ayn
Legend

What I meant was, often people will ask a question about how to make Splunk understand something, and often a key to answering that question is to formulate exactly how one would make Splunk understand it. In your case, Splunk can't possibly understand how to meet your requirement if it's not fed enough information to do so. Full path will not be available when doing file uploads (not in Splunk, nor in any other webapp). Sorry.

0 Karma

gelica
Communicator

I don't really understand what you want to know.. I know how to extract the fields when I have the whole path as I get when using monitors.

The second part of your comment is my answer I guess, I was really hoping that there was an easy way to get around this.

0 Karma

Ayn
Legend

Can you formulate in human language how you would identify the fields you need when uploading a file? Full path will never be supplied in file uploads (this is not unique to Splunk) so it's hard to think of a workaround to that...

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!