Getting Data In

Is there any way in which we can download the apps from splunk base without having to manually download the tar file?

Explorer

Hi

We are planning to automate the Splunk application installation and configuration process for quicker provisioning.

In this scenario, our first step is to install the splunk app from CLI, for which we use this command:

./splunk install app https://splunkbase.splunk.com/app/1274/

However it gives an error saying:

Error during app install: failed to extract app from /opt/splunk/var/run/87b95d9a426d8ebd.tar.gz to /opt/splunk/var/run/splunk/bundle_tmp/91801e5fc0eab8b4: No such file or directory

Is there any way in which we can download the apps from splunkbase without having to manually download the tar file.

Building on @mabrafoo's answer, I wrote a standalone script to do this. It allows you to authenticate to Splunkbase and download an app without the need for a separate web browser. Once you have the app.tgz, you can use the standard ./splunk install app <filename> syntax.

https://github.com/tfrederick74656/splunkbase-download

0 Karma

Loves-to-Learn Lots


Hi 

 

I tried the script and got this:

0Warning: Remote filename has no length!

 

curl: (23) Failed writing body (0 != 16195)


I also tried forcing the specific URL of an app and got the same result.

Any ideas?

Thank you.

0 Karma

Hi @Luis_Torres,

First, are you specifying the sid and SSOID arguments (example values show below) when running download? You'll get an error message just like this if you don't specify them, if the values are incorrect, or if the session they refer to is expired. All Splunkbase downloads are authenticated, so it's mandatory to supply these. The sid value is case-sensitive alphanumeric, so it can be easy to mistake "0" "o" and "O", for example. The SSOID value should be all hexadecimal (0-9, a-f).

2020-07-27_8-01-05.png

It's also worth double-checking that you actually have permissions to write to the directory where you're saving to. By default, the script will write to your current directory.

If none of that works, can you let me know if this is happening for all apps, or only a single app? If the latter, can you let me know the App ID and App Version you're trying to download?

Thanks,
Tyler

0 Karma

Engager

Here is one way to do it. Use at your own risk.

curlfire will access the firefox cookies so that we can avoid the "please log in to download" message that curl would get.

For this example the app is the splunk add-on for Unix and Linux. The url says it is app 833. These instructions assume we know that the app id number is 833.

After running this command
username@computername:~/Downloads/splunk/curlfire-master$ ./curlfire "https://splunkbase.splunk.com/app/833/" | grep 833 | grep download | grep release

The output is
sb-href="/app/833/release/6.0.0/download/">
sb-href="/app/833/release/5.2.4/download/">

Now we know the Download URL for the latest version is
https://splunkbase.splunk.com/app/833/release/6.0.0/download/

Download the file using curlfire (see notes for curlfire chanages to make it work better below)
username@computername:~/Downloads/splunk/curlfire-master$ ./curlfire "https://splunkbase.splunk.com/app/833/release/6.0.0/download"

Output
curl: Saved to filename 'splunk-add-on-for-unix-and-linux_600.tgz'

Notes
In order to get the download to work properly, 3 flags were changed when curl was run in the curlfire bash file.

Before
curl -b "$curlcookies" "${args[@]}" ;
After
curl -O -J -L -b "$curlcookies" "${args[@]}" ;

option -O Write output to a local file name like the remote file
option -J Tell the -O option to use the filename found in the http header
option -L Follow redirects

Also, after looking at a random script that I downloaded from github I usually will change this.

!/bin/bash

to this.

!/bin/bash -x

in order to display all of the bash scripts commands and their expanded arguments.

And obviously don't forget to change your curl user agent to something common like "I ❤️ splunk."

SplunkTrust
SplunkTrust

In order to download Apps from Splunkbase you need to be signed on to Splunkbase. Are you doing anything to sign on?

Personally, I wouldn't recommend automatically installing things that are downloaded fresh off the internet. How do you know it doesn't break your environment?
I'd keep a local repository of known good / fixed versions and install automatically from there.

Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!