Splunk Search

Is it possible to get data in via python script or read file?

eholz1
Contributor

Hello Splunk Community

I have a python script that checks a certain family of cisco devices that tells me if the Device is UP or DOWN. The script is based on a csv file that has hostname and IP.

The file is not really subject to change, but can be changed easily if required. I wish I could use the Splunk SNMP module, but I need some sort of API key (BaboonBones!??!)

I can use the script outside of splunk to create a “log” file then have splunk read the file. Maybe that is the best way, I am wondering if it is worthwhile to try to find the splunk python splunklib.client module and use it to send data, etc.

I am open to suggestions.

Thanksl,

eholz1

Labels (1)
0 Karma
1 Solution

starcher
Influencer

Another option is if you want to run the python outside of Splunk for some reason. Send your data in via HTTP event collector. 

https://github.com/georgestarcher/Splunk-Class-httpevent

View solution in original post

yuanliu
SplunkTrust
SplunkTrust

Another alternative is REST API (Endpoints reference list, see under receiv ers/).  Some pros and cons for your consideration.

 ProCon
Scripted inputSimple, arguably the lowest costEvery event carries the name of Spunk server/forwarder as host value
HECAllows setting of various meta data for each event, such as hostSpecialized interface, some setup
REST API, e.g., Allows setting of various meta data for each event, such as hostRequires authentication
File dump-ingestionSimple to implement, can set host field per event via path/file nameIf use path/file name for event, make sure each batch does not contain duplicate path/file name.

eholz1
Contributor

Hello All,

Thanks for the tips. I managed to get an HEC set up on the Splunk indexer.

I can go to a remote computer and do a curl command like this, which does send data to the indexer:

curl -k https://Indexer:8088/services/collector/event -H "Authorization: Splunk 6959a730-556f-4d91-6d94-a6f63fdfb72e" -d '{"event": "amazing transfer of hello world"}'

I am attempting to use a python program with the "requests" module imported, and it seem I need the urllib3 module as well.

I am using json for my header, and json for my data

header { "Authorization": "Splunk <token code from the HEC>" }

et = {"event": "UP hello world, etc" }

my request is this:

req = requests.post(url,headers=header,data=et,verify=False)

This fails with 400, Bad Request.

What am I missing here?

 

thanks

eholz1

 

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

Why do you need JSON for auth header?  As your cURL command demonstrated, it should be a colon-separated key-value pair.

0 Karma

starcher
Influencer

Another option is if you want to run the python outside of Splunk for some reason. Send your data in via HTTP event collector. 

https://github.com/georgestarcher/Splunk-Class-httpevent

eholz1
Contributor

Hello All,

Thanks for all the replies Yes, I used the HEC, and finally got everyting working.

I am using  the request module to send the data to splunk. The tricking part was  making the data value a STRING. The authorization goes through fine in the "json" format.

 

Thanks for the support

eholz1

0 Karma

eholz1
Contributor

Wow two good ideas. I will try it  out. One of my main questions would be monitoring a flle or using a

script - is one method better that the othe?

 

In my case, I might have to try the http event collector.

 

Thanks,

eholz

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Getting data into Splunk from a Python script is easy.  Run the script as a Splunk scripted input (Settings->Data inputs->scripts).  Anything the script writes to stdout will be indexed automatically - no client module needed.

---
If this reply helps you, Karma would be appreciated.

eholz1
Contributor

Hello richgalloway,

Again thanks for the tip here, both replies are VERY helpful, If there is a way to give you each 20 karmas

I would.

I will try the script method as well,

eholz1

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...