topic Re: Getting data in via python script or read file? in Splunk Search

Is it possible to get data in via python script or read file?

eholz1 — Sun, 27 Nov 2022 15:48:35 GMT

Hello Splunk Community

I have a python script that checks a certain family of cisco devices that tells me if the Device is UP or DOWN. The script is based on a csv file that has hostname and IP.

The file is not really subject to change, but can be changed easily if required. I wish I could use the Splunk SNMP module, but I need some sort of API key (BaboonBones!??!)

I can use the script outside of splunk to create a “log” file then have splunk read the file. Maybe that is the best way, I am wondering if it is worthwhile to try to find the splunk python splunklib.client module and use it to send data, etc.

I am open to suggestions.

Thanksl,

eholz1

Re: Getting Data In - via python script or read file

richgalloway — Fri, 18 Nov 2022 15:42:45 GMT

Getting data into Splunk from a Python script is easy. Run the script as a Splunk scripted input (Settings->Data inputs->scripts). Anything the script writes to stdout will be indexed automatically - no client module needed.

Re: Getting Data In - via python script or read file?

starcher — Fri, 18 Nov 2022 16:07:25 GMT

Another option is if you want to run the python outside of Splunk for some reason. Send your data in via HTTP event collector.

https://github.com/georgestarcher/Splunk-Class-httpevent

Re: Getting Data In - via python script or read file?

eholz1 — Mon, 21 Nov 2022 17:55:22 GMT

Wow two good ideas. I will try it out. One of my main questions would be monitoring a flle or using a

script - is one method better that the othe?

In my case, I might have to try the http event collector.

Thanks,

eholz

Re: Getting Data In - via python script or read file

eholz1 — Mon, 21 Nov 2022 17:57:33 GMT

Hello richgalloway,

Again thanks for the tip here, both replies are VERY helpful, If there is a way to give you each 20 karmas

I would.

I will try the script method as well,

eholz1

Re: Getting data in via python script or read file?

yuanliu — Tue, 22 Nov 2022 01:06:17 GMT

Another alternative is REST API (Endpoints reference list, see under receiv ers/). Some pros and cons for your consideration.

	Pro	Con
Scripted input	Simple, arguably the lowest cost	Every event carries the name of Spunk server/forwarder as host value
HEC	Allows setting of various meta data for each event, such as host	Specialized interface, some setup
REST API, e.g., /receivers/simple	Allows setting of various meta data for each event, such as host	Requires authentication
File dump-ingestion	Simple to implement, can set host field per event via path/file name	If use path/file name for event, make sure each batch does not contain duplicate path/file name.

Re: Getting data in via python script or read file?

eholz1 — Thu, 24 Nov 2022 03:01:31 GMT

Hello All,

Thanks for the tips. I managed to get an HEC set up on the Splunk indexer.

I can go to a remote computer and do a curl command like this, which does send data to the indexer:

curl -k https://Indexer:8088/services/collector/event -H "Authorization: Splunk 6959a730-556f-4d91-6d94-a6f63fdfb72e" -d '{"event": "amazing transfer of hello world"}'

I am attempting to use a python program with the "requests" module imported, and it seem I need the urllib3 module as well.

I am using json for my header, and json for my data

header { "Authorization": "Splunk <token code from the HEC>" }

et = {"event": "UP hello world, etc" }

my request is this:

req = requests.post(url,headers=header,data=et,verify=False)

This fails with 400, Bad Request.

What am I missing here?

thanks

eholz1

Re: Getting data in via python script or read file?

yuanliu — Thu, 24 Nov 2022 22:07:41 GMT

Why do you need JSON for auth header? As your cURL command demonstrated, it should be a colon-separated key-value pair.

Re: Getting Data In - via python script or read file?

eholz1 — Mon, 28 Nov 2022 15:32:07 GMT

Hello All,

Thanks for all the replies Yes, I used the HEC, and finally got everyting working.

I am using the request module to send the data to splunk. The tricking part was making the data value a STRING. The authorization goes through fine in the "json" format.

Thanks for the support

eholz1