Getting Data In

How to write a Scripted input directly to lookup?


I have a requirement where I need to make an API call and write the data to a lookup file that I can use locally. The API calls returns data in a CSV format. 


Previously, I used the Ad-on builder to create a python script that would make make the API request and index this data. However, I have a new requirement to skip the index entirely and write to a local lookup on the search head. The Ad-on builder wont help as it only shows examples of how to write the data to an index.


Thank you!

Labels (3)
Tags (4)
0 Karma

Super Champion

I would suggest using a custom Python command with the help of Splunklib as input is not recommended on the search head. Here is some parts of the code:




filename =
chunked = true


import os
import sys
import csv

from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators

HEADERS = ['ip','mac','hostname']   # change this as per your need
LOOKUP_NAME = 'my_lookup.csv'

class LookupGen(GeneratingCommand):
    def get_data_from_your_data_source(self):
        pass # write your logic to fetch the data here

    def update_lookup_file(self, lookup_file_path, data):
        with open(lookup_file_path, 'w') as f:
            csv_writer = csv.writer(f)

    def generate(self):
        data = self.get_data_from_your_data_source()
        lookup_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
                                            os.path.join('lookups', LOOKUP_NAME))
        self.update_lookup_file(lookup_path, data)

dispatch(LookupGen, sys.argv, sys.stdin, sys.stdout, __name__)



You will need to add this python file into your bin folder along with splunklib (Python SDK for Splunk) -


And you can schedule a search/report using this command at regular internal.


You don't need the Add-on Builder to do that.  You already have a Python script so just replace the part the indexes the data with a few lines of code to write it to ../lookups/mylookup.csv (or whatever you want to call the file).

If this reply helps you, Karma would be appreciated.
0 Karma


interesting. So should I use the Ad-On builder but at the very end of the script have it overwrite the lookup table? 

For background, the reason why I used the ad-on builder was because I was getting really confused with the authentication and safely pulling the API keys out of the passwords.conf. So the ad-on builder really helped with retrieval of secrets. 

Are you suggesting overwriting the lookup using the REST endpoint? If so, how do i do that without authenticating ? I see a lot of the curl commands require you to pass admin credentials; however, i dont want to hardcode any creds in my script.

0 Karma


AoB helps with the hard parts.  Writing data to a disk file is not a hard part.  Because scripted inputs run on the Splunk server, they have access to the file system there.  Just use normal pythonic methods for opening and writing to a text file.  You don't need REST.

If this reply helps you, Karma would be appreciated.
0 Karma


The input itself (at least with the input functionality) cannot write to a lookup.

You need a script that manipulates lookup using REST API. It has nothing to do with indexing.

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...