Is there Rest API for splunkbase to get list of all apps and version details?

[AnilPujar]

Are there any APIs for Splunkbase, I want to get the list of all apps available in Splunkbase with the below-mentioned information.

1. splunk app name

2. splunk folder name

3. app version

4. compatibility (like the app is compatible with Splunk version 7/8/9)

5. CIM compatibility

[rmccullagh]

I know this is a slightly older post but wanted to mention I've written an App & Add-on that does this quite easily! Please check it out and let me know if you have any feedback.

https://splunkbase.splunk.com/app/6665

[michael_vi]

Our Splunk instance has no internet access, so I'm trying to get a list of Splunk apps from splunkbase in JSON format and to compare the results via lookup to installed apps in Splunk.

The problem is that via API I can get info for specific app bit not for all of the apps in splunkbase.

https://splunkbase.splunk.com/api/v1/app/2919/

uid	2919
appid	"analysis_of_splunkbase_apps"
title	"Analysis Of SplunkBase Apps for Splunk"
created_time	"2022-06-29T22:21:12+00:00"
published_time	"2022-06-29T22:21:12+00:00"
updated_time	"2022-11-12T18:36:43+00:00"
type	"app"
license_name	"Splunk Software License Agreement"
license_url	"https://www.splunk.com/en_us/legal/splunk-software-license-agreement.html"
description	"After being asked for a list of Splunk Apps in a spreadsheet a few times, I found a need to build this App.  This App provides a simple dashboard with App stats and allows you to search for Splunk Apps within Splunk. It was also designed to work if you are offline, as long as you have been online once to collect data.\n\nThis App can be used to determine which Apps are certified to work with Splunk Cloud, Hunk, ES, etc.  \n\nThis is just one example of how you can use Splunk to analyze web site content.\n\n*NOTE: Firefox does not seem to like loading he dynamic menus, please use any other major browser."
access	"unrestricted"
appinspect_passed	true
path	"https://splunkbase.splunk.com/app/2919/"
install_method_distributed	"assisted"
install_method_single	"simple"
download_count	6015
install_count	0
archive_status	"live"
is_archived	false
fedramp_validation	"no"

https://splunkbase.splunk.com/api/v1/app/2919/release/

	
0	
id	26397
app	2919
name	"5.4.3"
product	
name	"splunk"
display_name	"Splunk"
release_notes	"Changes to the Splunkbase API have been reflected in our collection script.  \nLocal audit bug resolved."
CIM_versions	[]
product_versions	
0	"9.1"
1	"9.0"
2	"8.2"
3	"8.1"
4	"8.0"
5	"7.3"
6	"7.2"
7	"7.1"
8	"7.0"
created_datetime	"2022-06-29T22:21:12.495737Z"
published_datetime	"2022-06-29T22:21:12.495763Z"
size	217239
filename	"analysis-of-splunkbase-apps-for-splunk_543.tgz"
platform	"independent"
install_method_single	"simple"
install_method_distributed	"assisted"
fedramp_validation	"no"
cloud_compatible	true

So, my question is, how to get all apps list from splunkbase in JSON format

[richgalloway]

The API endpoint https://splunkbase.splunk.com/api/v1/app/ will return 25 apps by default.  Wrap it in a script to specify successive values for the ?offset parameter to collect all apps.

https://splunkbase.splunk.com/api/v1/app/?offset=25&limit=100

[AnilPujar]

this api don't have app version 

[michael_vi]

Is there any way to get the same # of rows (100) for the release URL, and not to do it 1 by 1?

https://splunkbase.splunk.com/api/v1/app/<UUID>/release/ 

[lanzykevin]

@michael_vi You happen to get this list at all? I may be needing it in the near future so just trying to get a head of it.

[michael_vi]

import requests

url = "https://splunkbase.splunk.com/api/v1/app/"
limit = 1

url2= "https://splunkbase.splunk.com/api/v1/app/"

with open(r"C:\Users\denis.zarfin\PycharmProjects\pythonProject2\main.txt", 'w') as f:
    f.write("name" + ", " + "uid" + ", " + "title" + ", " +'\n')

offset = -1
all_numbers = []
try :
    while True:
        offset += 1
        try:
            response = requests.get(url, params={"limit": limit, "offset": offset})
            data = response.json()
            for i in data["results"]:
                url2 = str(url2) + str(i["uid"]) + "/release/"
                response2 = requests.get(url2)
                data2 = response2.json()
                data2 = data2[0:1]
                for j in data2:
                    a = str(j["name"])
                    b = str(i["uid"])
                    c = str(i["title"])
                    with open(r"C:\Users\denis.zarfin\PycharmProjects\pythonProject2\main.txt", 'a') as f:
                        f.write(a + ", " + b + ", " + c + ", " +'\n')
            url2 = "https://splunkbase.splunk.com/api/v1/app/"
        except:
            pass
        print(offset, a, b, c)
        if offset > 2700:
            break
except:
    pass
print("ok")

That one exports the results to CSV... but it's not that good.

In the end I want to get 2 JSONs

I was able to do it "manually" with:

import json
import requests

result = []
for app_id in range(0, 1, 1):
    url = f'https://splunkbase.splunk.com/api/v1/app/?offset={app_id}&limit=1'

    data = requests.get(url).json()

    print(f'Name: {data["results"]['uid']}')

[lanzykevin]

@michael_vi First, thank you for the help! I was able to get this script updated here and it puts it in a alphabetical list. Thank you again

import requests

import json

# Splunkbase API endpoint to get a list of all apps

splunkbase_api_url = "https://splunkbase.splunk.com/api/v1/app/"

# Initialize an empty list to store all apps

all_apps = []

# Initialize offset and batch size

offset = 0

batch_size = 25  # You can adjust this if needed

while True:

    # Construct the URL with the current offset

    params = {"offset": offset}

    response = requests.get(splunkbase_api_url, params=params)

    if response.status_code == 200:

        # Parse the JSON response to access the list of apps

        apps = response.json()

        # Add the retrieved apps to the list

        all_apps.extend(apps['results'])  # Use ['results'] to get the apps

        # Calculate the total number of apps from the response

        total_apps = apps['total']

        # If we have collected all apps, break the loop

        if offset >= total_apps:

            break

        # Increase the offset for the next request

        offset += batch_size

    else:

        print("Failed to retrieve apps. Status code:", response.status_code)

        break

# Extract 'appid' (app names) from the 'results'

app_names = [app['appid'] for app in all_apps]

# Sort the app names in alphabetical order

app_names_sorted = sorted(app_names)

# Create a dictionary with the sorted app names as a list

app_data = {"app_names": app_names_sorted}

# Save the app data to a JSON file as before

output_file_path = "splunkbase_apps.json"

with open(output_file_path, 'w') as json_file:

    json.dump(app_data, json_file, indent=4)  # Use indent to format JSON

print(f"App data has been saved to {output_file_path}")

[richgalloway]

The same script the increments the offset argument also can extract the UUID and send it the https://splunkbase.splunk.com/api/v1/app/<UUID>/release/ endpoint. 

[mihai_wrld]

@richgalloway 

Did you or anyone else in this post ever end up creating the full script to get recent versions from  "https://splunkbase.splunk.com/api/v1/app/<UUID>/release/" Endpoint ? 🤔
 

[AnilPujar]

@richgalloway 

Thank you .

[richgalloway · Accepted solution]

Yes, there is an API for splunkbase.  See https://dev.splunk.com/enterprise/reference/splunkbase/

There is also an app that does what you seek.  See Analysis of Splunkbase Apps for Splunk (https://splunkbase.splunk.com/app/2919).