Splunk Search

How to Script a Lookup in python

alexantao
Path Finder

Hello,

I have a LDAP server and a proxy that logs the users login-name into splunk. I need to make a lookup that gets the username (field user_id in splunk), searches on my LDAP server and convert then to the realname, adding another field in splunk on a search.
I'm trying to make a script for this lookup, but I cannot figure it out how it really works. I've studied the external lookup example (and another one I found on Internet), but it is not clear enough on how it really works to send the results to splunk.

Could somebody help me ? Below is a simple code I made (based on external_lookup.py) that supposelly) simply had to get the user_id and set a string ("Usuario Temp") to it. It does not searches on LDAP yet. But I couldn't even make it return the results to SPLUNK.

If somebody could point me to a tutorial or a document explaining how are the structures SPLUNK expect, and a class reference it would also be helpful, because official documentation are not clear on how everything works.
Thanks in advance.

Here is my code:

    #!/usr/bin/python

import sys
import csv

FIELDS = [ "user_id", "nome_usuario" ]

def main():

        infile = sys.stdin
        outfile =  sys.stdout

        entrada =  csv.DictReader(infile)
        header = entrada.fieldnames

        saida = csv.DictWriter(outfile, fieldnames=entrada.fieldnames)
        saida.writeheader()

        ##########

        for valor_atual in entrada:
                valor_atual["user_id"] = "Usuario Temp"
                saida.writerow(valor_atual)

main()
0 Karma

musskopf
Builder

Go to your $SPLUNK_HOME/etc/system/bin and look for the script "external_lookup.py". That's a very simple example.

For you to understand, the lookup script receives the data via PIPE in CSV format and return the same data with the lookup fields populated, for example (on Linux):

Create a file /tmp/test.csv with the content:

host,ip
google.com,
microsoft.com,

Run the commands (adapt to your environment):

export SPLUNK_HOME=/apps/splunk
export PYTHONPATH=$SPLUNK_HOME/lib/python2.7

cat /tmp/test.csv | $SPLUNK_HOME/bin/python $SPLUNK_HOME/etc/system/bin/external_lookup.py host ip

You should see an CSV output like that:

host,ip
google.com,131.242.32.38
google.com,131.242.32.42
(...)
google.com,131.242.32.37
microsoft.com,134.170.185.46
microsoft.com,134.170.188.221

The output above is how the lookup works... it injects a CSV with all fields, including the empty ones and your script will be responsible to handle it and fill the gaps.

Now if you decide to use any Python library, make sure it exists inside $SPLUNK_HOME/lib/python2.7, and Splunk will run all Python scripts using it's own Python and not the Python the your OS. If you can't find a required library you might need to compile it using an Python 2.7 and copy there... you might get around by copying the OS one there, but I already have trouble doing that as my OS uses Python 2.6.

Hope it helps!

keeblerh
New Member

musskopf, I do not have access to the server to look at the simple py lookup example. Can the contents be posted? We are working with an evaluation copy and the installers are not very available. Thanks.

0 Karma

musskopf
Builder

Good stuff! Don't forget to accept the answer to help others finding this information.

Btw, have a look on this App: http://apps.splunk.com/app/1151/ is says Active Directory but you can use with any LDAP and might do what you're after.

Cheers!

0 Karma

alexantao
Path Finder

Thank you! Knowing how splunk treats input and output and how to test inside it was important. Now It was a simple matter on understanding csv class reference.
Now I am getting some output and will work on LDAP stuff.

Get Updates on the Splunk Community!

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...