How to I use the REST API to create multiple assets in Splunk Phantom?

[kevinh_splunk]

This article provides an example of how to use the Splunk Phantom REST API to create multiple assets. This may be useful if you have a large number of similar asset types, such as firewalls, Windows servers, or vSphere servers. See Using the REST API reference for Splunk Phantom for more information.

To create as asset in Splunk Phantom using the REST API, post a JSON object to a specific URL on the Splunk Phantom server.  TO see the contents of the JSON, you can manually create an example asset, then export the JSON using the API. The following script (tested with Python 2.7) exports all of your assets to JSON files in the current directory. Replace the host, username, and password with the actual values from your own environment:

"""
export_assets.py

This script will go through the list of Assets in a Phantom server and create a .json file for each in the current directory
"""
import requests, json

host = '10.16.0.201' #IP address or hostname of the Phantom server
username = 'admin'
password = 'password'
verifycert = False #Default to not checking the validity of the SSL cert, since Phantom defaults to self-signed
maxpages = 100 #Assume there are no more than 100 pages of results, to protect against an accidental loop
baseurl='https://' + host + '/rest/asset?page=' #The base URL for a list of all assets, by page number
currentpage = 0

if not verifycert:
    requests.packages.urllib3.disable_warnings() #If disabling SSL cert check, also disable the warnings
assetids = []
while True:
    currenturl = baseurl + str (currentpage)
    r = requests.get(currenturl, auth=(username, password), verify = verifycert)
    results = r.json()
    pages = int (results['num_pages']) #Get the total number of pages, so we know how many to read
    currentpage = currentpage + 1
    for asset in results['data']:
            assetids.append (asset['id']) #Save the asset ID for each asset, so we can fetch each one
    if (currentpage >= pages) or (currentpage >= maxpages): #If we have read all the pages or hit the limit, stop
            break

baseurl='https://' + host + '/rest/asset/' #The base URL for an individual asset, by Asset ID
for assetid in assetids:
    currenturl = baseurl + assetid #Append the Asset ID to the base URL
    r = requests.get(currenturl, auth=(username, password), verify = verifycert)
    results = r.json()
    filename = results['name'] + '.json' #Open a file for writing with the asset name for the filename and .json for an extension
    file = open(filename, 'w')
    file.write (json.dumps(results, indent=2, sort_keys=True)) #Write out the JSON in mostly-human-readable format

Below is an example of the JSON file produced by the script:

{
  "configuration": {
    "ingest": {},
    "password": "vN9ExjcSq1GhkTzOmwwDTA==",
    "server": "10.16.0.151",
    "username": "root"
  },
  "description": "",
  "disabled": false,
  "id": "3a15ae6f-1013-4143-82c1-a58c30a27bbb",
  "name": "labesxi1",
  "primary_owner": {
    "user_ids": [],
    "voting": 0
  },
  "product_name": "vSphere",
  "product_vendor": "VMware",
  "product_version": "",
  "secondary_owner": {
    "user_ids": [],
    "voting": 0
  },
  "tags": [
    ""
  ],
  "token": null,
  "type": "virtualization",
  "version": 1
}

You can edit the JSON file and change the fields as needed, such as the name, server, and password.  After making the desired changes, use the following script to post the JSON file to the correct URL:

"""
create-asset.py
This script takes a JSON file representing a Phantom Asset, and creates the Asset on the server
"""
import requests, json, sys

host = '10.16.0.201' #IP address or hostname of the Phantom server
username = 'admin'
password = 'password'
verifycert = False #Default to not checking the validity of the SSL cert, since Phantom defaults to self-signed
baseurl='https://' + host + '/rest/asset' #The base URL for posting to create an Asset

if len(sys.argv) < 2:
    print 'Usage: ' + sys.argv[0] + ' [filename]'
    sys.exit (1)

assetfilename = sys.argv[1]

with open(assetfilename) as assetfile:
    assetjson = json.load(assetfile)

if not verifycert:
    requests.packages.urllib3.disable_warnings() #If disabling SSL cert check, also disable the warnings

#Post the JSON to the URL
r = requests.post(baseurl, data = json.dumps(assetjson), auth=(username, password), verify = verifycert)
print json.dumps(json.loads(r.text), indent=2, sort_keys=True) #Print the results from the server

When you export an asset using the API, you get all the fields saved for that asset, including some internal fields that you don't need to specify when you create an asset.

Field	Description
id	Each asset is assigned an ID when the asset is created. If you use the JSON file to create a new asset, the existing ID is ignored.
password	If you use the JSON file to create a new asset, the existing encrypted password causes the creation of the new asset to fail authentication. You must replace the encrypted password with a clear text password.

The minimum number of fields required to create an asset depends on the asset type. Some assets may only require a name, vendor, and product, while others may also require several additional fields. The following example shows the minimum field in the JSON for a vSphere asset:

{
  "configuration": {
    "server": "10.16.0.151",
    "username": "root",
    "password": "password"
  },
  "name": "labesxi1",
  "product_name": "vSphere",
  "product_vendor": "VMware"
}

You can edit this file to change the name and IP address, the post it to create a new asset. However, this requires you to edit and post the file once for each asset.

An alternative for creating a large number of similar assets is to use a CSV file. For example:

"name","product_name","product_vendor","configuration:server","configuration:username","configuration:password"
"labesxi1","vSphere","VMware","10.16.0.151","root","password"
"labesxi2","vSphere","VMware","10.16.0.152","root","password"
"labesxi3","vSphere","VMware","10.16.0.153","root","password"
"labesxi4","vSphere","VMware","10.16.0.154","root","password"
"labesxi5","vSphere","VMware","10.16.0.155","root","password"
"labesxi6","vSphere","VMware","10.16.0.156","root","password"
"labesxi7","vSphere","VMware","10.16.0.157","root","password"
"labesxi8","vSphere","VMware","10.16.0.158","root","password"
"labesxi9","vSphere","VMware","10.16.0.159","root","password"
"labesxi10","vSphere","VMware","10.16.0.160","root","password"

We can write a script that will create all of these assets at once. The column names in the first line are very important for the script, as they represent the JSON variable names. 

• The name, product_name, and product_vendor parameters are common to every asset.
• The server, username, and password parameters are specific to the asset type and are in a sub-list named configuration.  Hence, the server is referred to as configuration:server.

The following script reads the CSV and creates the assets:

"""
create_assets_from_csv.py
This script takes a CSV file containing parameters needed to create multiple Phantom Assets
"""
import requests, json, sys, csv

host = '10.16.0.201' #IP address or hostname of the Phantom server
username = 'admin'
password = 'password'
verifycert = False #Default to not checking the validity of the SSL cert, since Phantom defaults to self-signed
baseurl='https://' + host + '/rest/asset' #The base URL for posting to create an Asset

if len(sys.argv) < 2:
    print 'Usage: ' + sys.argv[0] + ' [filename]'
    sys.exit (1)

csvfilename = sys.argv[1]

header = []
rows=[]
with open(csvfilename) as csvfile:
    reader = csv.reader(csvfile)
    first_row = True
    for row in reader:
            if first_row:
                    header = row #save the header row
                    #got header
                    first_row = False
                    continue
            if not row:
                    continue
            rows.append(row) #save the data rows into a list

if not verifycert:
    requests.packages.urllib3.disable_warnings() #If disabling SSL cert check, also disable the warnings

for row in rows: #loop through the data rows
    asset = {}
    asset['configuration'] = {}
    for i, column in enumerate(header): #loop again for each header column
            print column + " = " + row[i]
            if column.startswith('configuration:'): #if the column name starts with configuration:, stick it under the configuration list
                    subcolumn = column.split('configuration:',1)[1]
                    asset['configuration'][subcolumn] = row[i]
            else:
                    asset[column] = row[i] #otherwise add the variable to the top level of the JSON
    print json.dumps(asset, indent=2, sort_keys=True) #pretty-print the JSON we made
    #post the JSON
    r = requests.post(baseurl, data = json.dumps(asset), auth = (username, password), verify = verifycert)
    #pretty-print the JSON result from the server
    print json.dumps(json.loads(r.text), indent=2, sort_keys=True)
    print

For a different asset type, create a separate CSV with the correct header line for that asset type, and one row for each asset.