topic Re: Use Python to upload file into Splunk in Splunk Dev

Use Python to upload file into Splunk

ericchaucl — Mon, 04 Sep 2017 09:50:39 GMT

I follow the instructions in here http://dev.splunk.com/view/python-sdk/SP-CAAAEE6 and just want to upload 1 single file into Splunk using Python with codes like this:

myindex = service.indexes["test_index"]
uploadme = "test.csv"
myindex.upload(uploadme);

I simply put the test.csv file together with the source code. However, it always show the error:
splunklib.binding.HTTPError: HTTP 400 Bad Request -- unable to open file: path='test.csv' error='The system cannot find the file specified.'

I've also tried to specify the whole path like this but also gives the same error:
uploadme = "c:/test/test.csv"

How should I specify the filename (in Windows)? Or where should I put the file?

P.S. I've already built the Splunk connection in Python and can run search properly.

Re: Use Python to upload file into Splunk

mumblingsages — Mon, 04 Sep 2017 13:27:23 GMT

Are you trying to upload the file to a remote server? If so, the default service configuration for Splunk is such that it cannot reference files that are not stored locally. In such a case the file path you are specifying must be local to the server itself.

Re: Use Python to upload file into Splunk

woodcock — Mon, 04 Sep 2017 15:17:25 GMT

Why bother with any code at all? Splunk already has a CLI for this called add oneshot:
http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorfilesanddirectoriesusingtheCLI

Re: Use Python to upload file into Splunk

ericchaucl — Tue, 05 Sep 2017 02:27:17 GMT

Thanks! If the default service Splunk configuration cannot reference files that are not stored locally, can I update the configuration to allow this?

Re: Use Python to upload file into Splunk

ericchaucl — Tue, 05 Sep 2017 02:36:01 GMT

I'm writing a Python program to analyze some external raw data and want to export the results into Splunk for further handling/reporting. Would be perfect if I can control the whole workflow in Python directly.

Thanks for the advise and may have a try on that.

Re: Use Python to upload file into Splunk

mumblingsages — Tue, 05 Sep 2017 15:23:06 GMT

I believe there is but I haven't had time to fully investigate that as part of my development. Instead, I've created network share on the remote host that my python script has access too (and nothing else) and it copies it's to there. Then in my upload call via python I specify the local path (not the share name) on the remote host.

Re: Use Python to upload file into Splunk

ericchaucl — Wed, 06 Sep 2017 01:13:44 GMT

Good idea and it works. Thanks!

Re: Use Python to upload file into Splunk

tolaram — Tue, 02 Apr 2019 03:48:42 GMT

Did this work for you ?

Re: Use Python to upload file into Splunk

ericchaucl — Tue, 02 Apr 2019 06:44:40 GMT

Busy in working on something else and have put down this issue for a long time .....

This approach could work for my case to build a prototype only.

Re: Use Python to upload file into Splunk

ericchaucl — Tue, 02 Apr 2019 06:44:48 GMT

Busy in working on something else and have put down this issue for a long time .....

This approach could work for my case to build a prototype only.