Splunk Search

Is it possible to get data in via python script or read file?

eholz1
Communicator

Hello Splunk Community

I have a python script that checks a certain family of cisco devices that tells me if the Device is UP or DOWN. The script is based on a csv file that has hostname and IP.

The file is not really subject to change, but can be changed easily if required. I wish I could use the Splunk SNMP module, but I need some sort of API key (BaboonBones!??!)

I can use the script outside of splunk to create a “log” file then have splunk read the file. Maybe that is the best way, I am wondering if it is worthwhile to try to find the splunk python splunklib.client module and use it to send data, etc.

I am open to suggestions.

Thanksl,

eholz1

Labels (1)
0 Karma
1 Solution

starcher
SplunkTrust
SplunkTrust

Another option is if you want to run the python outside of Splunk for some reason. Send your data in via HTTP event collector. 

https://github.com/georgestarcher/Splunk-Class-httpevent

View solution in original post

yuanliu
SplunkTrust
SplunkTrust

Another alternative is REST API (Endpoints reference list, see under receiv ers/).  Some pros and cons for your consideration.

 ProCon
Scripted inputSimple, arguably the lowest costEvery event carries the name of Spunk server/forwarder as host value
HECAllows setting of various meta data for each event, such as hostSpecialized interface, some setup
REST API, e.g., Allows setting of various meta data for each event, such as hostRequires authentication
File dump-ingestionSimple to implement, can set host field per event via path/file nameIf use path/file name for event, make sure each batch does not contain duplicate path/file name.

eholz1
Communicator

Hello All,

Thanks for the tips. I managed to get an HEC set up on the Splunk indexer.

I can go to a remote computer and do a curl command like this, which does send data to the indexer:

curl -k https://Indexer:8088/services/collector/event -H "Authorization: Splunk 6959a730-556f-4d91-6d94-a6f63fdfb72e" -d '{"event": "amazing transfer of hello world"}'

I am attempting to use a python program with the "requests" module imported, and it seem I need the urllib3 module as well.

I am using json for my header, and json for my data

header { "Authorization": "Splunk <token code from the HEC>" }

et = {"event": "UP hello world, etc" }

my request is this:

req = requests.post(url,headers=header,data=et,verify=False)

This fails with 400, Bad Request.

What am I missing here?

 

thanks

eholz1

 

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

Why do you need JSON for auth header?  As your cURL command demonstrated, it should be a colon-separated key-value pair.

0 Karma

starcher
SplunkTrust
SplunkTrust

Another option is if you want to run the python outside of Splunk for some reason. Send your data in via HTTP event collector. 

https://github.com/georgestarcher/Splunk-Class-httpevent

eholz1
Communicator

Hello All,

Thanks for all the replies Yes, I used the HEC, and finally got everyting working.

I am using  the request module to send the data to splunk. The tricking part was  making the data value a STRING. The authorization goes through fine in the "json" format.

 

Thanks for the support

eholz1

0 Karma

eholz1
Communicator

Wow two good ideas. I will try it  out. One of my main questions would be monitoring a flle or using a

script - is one method better that the othe?

 

In my case, I might have to try the http event collector.

 

Thanks,

eholz

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Getting data into Splunk from a Python script is easy.  Run the script as a Splunk scripted input (Settings->Data inputs->scripts).  Anything the script writes to stdout will be indexed automatically - no client module needed.

---
If this reply helps you, Karma would be appreciated.

eholz1
Communicator

Hello richgalloway,

Again thanks for the tip here, both replies are VERY helpful, If there is a way to give you each 20 karmas

I would.

I will try the script method as well,

eholz1

0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...