Carbon footprint for a splunk search

[JonasLind]

Hi,

We are about to start up a new project where the project manager need to know the carbon footprint of the work done by Splunk.
How do I calculate that?

We will use DB connect to get data into Splunk (running once every hour) and also the same App to send some data to another tool (once every 24h). I guess that the daily impact will be about 50mb of data into Splunk, and about 10mb out from Splunk.

Is there some app that I can use for this, or does Splunk have some guidelines how to calculate this?

thanks
Jonas

[mitag]

I'd ask the manager for guidance. (To guide you is manager's job :))

• How is CF calculated for other services - or anything else - in the company? If there is a guide for that, then you'd use it for this specific purpose.
• If there isn't any guide - then you're essentially tasked with first creating one - creating a baseline, a framework, a process for calculating CF for anything and everything in your company or department. You would then want to ensure your manager understands that you can't do what he is asking w/o establishing such a framework. Only then (once there is a guide), you can use it for your specific purpose. Otherwise - like @bgraabek_splunk astutely posited - it is next to impossible to produce an accurate estimate. E.g. you may need to include the CF of the admin workstation from which your Splunk instance is managed prorated to time spent on Splunk... It would get even more complicated from there.

[nickhills]

It’s a nice idea, but there are so many (incalculable) variables that any value you quote will never stand up to scrutiny.

Carbon production is a huge concern, but if you are going to make a case that x product/service produces y in terms of C you need to be able to defend you calculations and assumptions.

The problem with trying to calculate this from computing services (which do not linearly scale with electricity consumption) is that anyone with a small amount of knowledge will lend your calculations little credibility and that is damaging to the concept of the intent.

The best you are likely to be able to achieve is to define (require) that your services are run from carbon neutral data centres. (AWS have I think 4).
Because AWS manage the energy usage in their facilities, they can ensure that in these calculations they are accurate.

You can then (by explaining your reasoning) articulate that your Splunk environment is carbon neutral, but anything else would in all likelihood be wild speculation.

My somewhat facetious comment above is not without merit if you want to make some calculations on your own organisation.
Canvas your Splunk users and ask them how many hours a day would they loose if they did not have the use of Splunk or had to run processes by hand. Convert this value across your org to a number of FTEs and calculate the C footprint of employing that many more people, their energy, resource and travel consumption.

It’s not a direct answer to your question, but it is a result you will be able to defend, and others will be able to understand.

[bgraabek_splunk]

It is impossible to figure out what the carbon footprint will be just based on the software you are using, how much electricity the process uses depends on the CPU, the disks, the power supplies (and all the other components) in the server where Splunk is installed. But here is a way to figure it out.

1. First, find out what the baseline electricity use of the Splunk server is over a given period of time (maybe 24 hours?) without running the processes you want to know the carbon footprint for.
2. Then run the processes for which you want to know what the carbon footprint is for the same period as you ran the Splunk server to discover the baseline use was. Again, measure the amount of electricity used during the measuring period.
3. Subtract the value you just measured from the baseline. If there is little difference, make the measuring period longer.
4. Find out how much carbon dioxide is produced per kWh by your electricity supplier. If your supplier has a lot of nasty old coal-fired power plants, the number is going to be high. If the supplier sources their electricity from lots of wind turbines, the number os going to be lower.
5. Now do the maths.

Lets assume that you find that the server uses 5 Wh of electricity more over a 24 hour period and that the carbon footprint of your supplier is 500 gram/kWh, the calculation would look something like this:
0.005 kWh * 365 days * 0.5 kg co2 = 0.9125 kg. That would be the carbon footprint for the process.

However, how precise should this number be? Is the Splunk data stored in a SAN? If it is, maybe you should also find out how much extra electricity the SAN is using. Is the Splunk server a single instance or in a cluster? What about the database the data is being pulled from? What about the electricity used to send data from the Splunk server to the user when they access it? Now you are involving switches, maybe also routers and firewalls. Is the server in Seattle and the user in New York? How much electricity do the ISPs use to get the data from your data center to the user?

[mitag]

.... and what about the carbon footprint of the fridge where all the milk is stored that Splunk Admin drinks during Splunk all-nighters? And... what about them cows?... 😉

[to4kawa]

In this case,
We need to consider not only Splunk itself, but also the environment in which it runs, I think.
I've learned a lot. thanks.

[JonasLind]

Thanks for the answers! I had to ask, because I didn't have any clue where to start.

However this is something that will be more and more relevant for the most companies in the future. So maybe an app to calculate these things would be nice?

I think the question is quite tricky, but it would be nice to be able to calculate the carbon footprint of any server based on some parameters, like cpu usage, memory usage, server type etc.

[nickhills]

I’d start from the other direction.

How much carbon would “not using Splunk” cost if you used graphite and paper to do the same reports.

That way you can argue Splunk is providing a carbon offset!!!

[richgalloway]

The Splunk software itself produces no carbon.

[nickhills]

🙂