I have a LDAP server and a proxy that logs the users login-name into splunk. I need to make a lookup that gets the username (field user_id in splunk), searches on my LDAP server and convert then to the realname, adding another field in splunk on a search.
I'm trying to make a script for this lookup, but I cannot figure it out how it really works. I've studied the external lookup example (and another one I found on Internet), but it is not clear enough on how it really works to send the results to splunk.
Could somebody help me ? Below is a simple code I made (based on external_lookup.py) that supposelly) simply had to get the user_id and set a string ("Usuario Temp") to it. It does not searches on LDAP yet. But I couldn't even make it return the results to SPLUNK.
If somebody could point me to a tutorial or a document explaining how are the structures SPLUNK expect, and a class reference it would also be helpful, because official documentation are not clear on how everything works.
Thanks in advance.
Here is my code:
#!/usr/bin/python import sys import csv FIELDS = [ "user_id", "nome_usuario" ] def main(): infile = sys.stdin outfile = sys.stdout entrada = csv.DictReader(infile) header = entrada.fieldnames saida = csv.DictWriter(outfile, fieldnames=entrada.fieldnames) saida.writeheader() ########## for valor_atual in entrada: valor_atual["user_id"] = "Usuario Temp" saida.writerow(valor_atual) main()
Go to your $SPLUNK_HOME/etc/system/bin and look for the script "external_lookup.py". That's a very simple example.
For you to understand, the lookup script receives the data via PIPE in CSV format and return the same data with the lookup fields populated, for example (on Linux):
Create a file /tmp/test.csv with the content:
host,ip google.com, microsoft.com,
Run the commands (adapt to your environment):
export SPLUNK_HOME=/apps/splunk export PYTHONPATH=$SPLUNK_HOME/lib/python2.7 cat /tmp/test.csv | $SPLUNK_HOME/bin/python $SPLUNK_HOME/etc/system/bin/external_lookup.py host ip
You should see an CSV output like that:
host,ip google.com,22.214.171.124 google.com,126.96.36.199 (...) google.com,188.8.131.52 microsoft.com,184.108.40.206 microsoft.com,220.127.116.11
The output above is how the lookup works... it injects a CSV with all fields, including the empty ones and your script will be responsible to handle it and fill the gaps.
Now if you decide to use any Python library, make sure it exists inside $SPLUNK_HOME/lib/python2.7, and Splunk will run all Python scripts using it's own Python and not the Python the your OS. If you can't find a required library you might need to compile it using an Python 2.7 and copy there... you might get around by copying the OS one there, but I already have trouble doing that as my OS uses Python 2.6.
Hope it helps!
musskopf, I do not have access to the server to look at the simple py lookup example. Can the contents be posted? We are working with an evaluation copy and the installers are not very available. Thanks.
Good stuff! Don't forget to accept the answer to help others finding this information.
Btw, have a look on this App: http://apps.splunk.com/app/1151/ is says Active Directory but you can use with any LDAP and might do what you're after.
Thank you! Knowing how splunk treats input and output and how to test inside it was important. Now It was a simple matter on understanding csv class reference.
Now I am getting some output and will work on LDAP stuff.