topic How to get Splunk to fetch JSON data from Enphase REST API? in Getting Data In

How to get Splunk to fetch JSON data from Enphase REST API?

emdeex — Fri, 08 Apr 2016 06:33:38 GMT

Hi All, I'm brand-new to Splunk, just found it...

Enphase (www.enphase.com) is a manufacturer of Solar PV Micro-Inverters and associated equipment. They are one of the more IT friendly manufacturers, and have a REST based API available to query installed equipment/solar installations. I've just got a new Enphase system installed, and I'd like to see if Splunk can query the data.

The REST API is detailed here: https://developer.enphase.com/docs

I've written a curl script to fetch the JSON data from my system, which works OK.

But can I make Splunk do the HTTP GET request itself, and schedule that?

Or do I have to get curl to do it, schedule a task, and get Splunk to read the file?

Thanks for any advice!

Re: How to get Splunk to fetch JSON data from Enphase REST API?

mnatkin_splunk — Tue, 29 Sep 2020 10:55:03 GMT

Hi there!

Funny you should ask, as I just started playing on this use case -- Splunking data from my Enphase system. The easiest way I know of to do what we want is to use the REST API Modular Input App -- see https://splunkbase.splunk.com/app/1546/#/details for details.

This will add a new data input method (you can browse from Settings -> Data -> Data Inputs) that provides a nice, simple UI for building the query. No need for curl scripts and cron jobs 😉

In my testing, I'm using the Enphase "stats" API call. Because Enphase wants time in epoch, you'll want to tweak the app's token.py ($SPLUNK_HOME/etc/apps/rest_ta/bin/token.py) to add a token for the current time (or some variant thereof, such as current time minus 5 minutes (the interval that Enphase supports for the stats command)) . In my case, I added the following:

 import calendar 
 def epochdate():
     # UTC format: timenow = datetime.datetime.utcnow()
     timenow = datetime.datetime.now()
     return calendar.timegm(timenow.timetuple())

Using "calendar" is the cleanest way I've found so far to do the conversion. I can then use the %epochdate% token in my config.

I am eager to hear how you fare with this project!

Happy Splunking!

Re: How to get Splunk to fetch JSON data from Enphase REST API?

mnatkin_splunk — Fri, 09 Sep 2016 17:51:25 GMT

It occurred to me that not everyone will be up on how to get going with API access for Enphase...

For those of you who are interested in this integration, be advised that there are a few steps required in order to enable API access:

0) Make sure API access is allowed and you've documented both your API username and your system ID.
- Navigate to https://enlighten.enphaseenergy.com, log in, then access the "Settings" by clicking on the down arrow next to your name
- You'll see your API ID under "API Settings"
- Your system ID is most easily distinguised in the URL that is presented to you when you log in... https://enlighten.enphaseenergy.com/pv/systems/{SYSTEM_ID}/overview

1) Navigate to https://developer.enphase.com/docs/quickstart.html for a brief overview of the API integration and operation
2) Open a Developer Account by registering at https://developer.enphase.com/signup
3) Define the "Application" on the Enphase site (in order to generate an App Key): https://developer.enphase.com/admin/applications
- You'll need to define the app tier you wish to use -- "Watt" is the free tier, allowing for up to 10k hits per month
4) Activate the app by navigating to the link provided when you define the app
5) Document the app key along with your API user ID
6) Test from the CLI before defining the REST API input
- From the CLI of your Splunk instance, run a simple curl, such as:

curl -v "https://api.enphaseenergy.com/api/v2/systems/{SYSTEM_ID}/summary?key={MY_APP_KEY}&user_id={MY_USER_ID}"

This should return an HTTP 1.1/200 response with JSON-formatted payload

Re: How to get Splunk to fetch JSON data from Enphase REST API?

sloshburch — Mon, 12 Sep 2016 17:42:35 GMT

This might also assist: https://splunkbase.splunk.com/app/2962

Re: How to get Splunk to fetch JSON data from Enphase REST API?

emdeex — Sat, 26 Nov 2016 05:54:33 GMT

Thanks for the help above...

I had a hack with it today, and using the REST API Modular Input, got it to query the Envoy-S every 60 seconds.

I found a good collection of JSON info on the Envoy-S device at the URL: http://192.168.1.100/production.json

There's a lot of info in there... (voltages, power, energy, power factor) ...so far I've only got basic stuff displaying... but it looks promising... the Envoy-S has consumption and production monitoring (and battery storage if you have it)...

I'd like to make a wall-mounted display of a similar dashboard eventually.

Re: How to get Splunk to fetch JSON data from Enphase REST API?

kozanic_FF — Mon, 11 Feb 2019 03:11:26 GMT

Hi Guys,

Not sure if you are still watching this post, but wondering if you might be able to share some of your inputs / props file details??

I've managed to use what you have shared so far to get to the point where I can connect and get data into Splunk from my enhpase system, but it's not in a usable state atm and struggling to get it to that point.

Appreciate anything you are willing to share.

Main issues I'm having atm are:

specifying the time range that I want to collect
Splitting the response into time blocks