Splunk Dev

How do I return AWS storage use by region/month -- only returns most recent month no matter what I try?

brockwebb
Explorer

earliest=-360d aws-description-resource( (aws_account_id="*") , (region="*") , "ec2_volumes") |convert timeformat="%Y-%m-%d %H:%M:%S" ctime(_time) AS c_time |timechart span=1months sum(size) as total_size by region

I can get this info using the right filters in AWS cost explorer, but drives me batty because I can't seem to do it in splunk... above is the query I'm running, and I've tried many variants on the theme of aggregating by month with no success.

Tags (1)
0 Karma
1 Solution

brockwebb
Explorer

Problem was a pre-defined macro:
"aws-description-resource" macro was set up to only look at current data within the last 55 minutes, so trying to use that macro for historical data was never going to work. Instead I modified the search calling out the index vs the macro and now the data is coming in as it appears it should.

Query:

((index="aws" OR index="default") sourcetype="aws:description" aws_account_id="" region="" source="*:ec2_volumes") | eventstats latest(_time) as latest_time | eval latest_time=relative_time(latest_time,"-55m") | dedup id sortby -_time | bin _time span=1mon@mon0 | stats sum(size) by _time, region

This works!

View solution in original post

0 Karma

brockwebb
Explorer

Problem was a pre-defined macro:
"aws-description-resource" macro was set up to only look at current data within the last 55 minutes, so trying to use that macro for historical data was never going to work. Instead I modified the search calling out the index vs the macro and now the data is coming in as it appears it should.

Query:

((index="aws" OR index="default") sourcetype="aws:description" aws_account_id="" region="" source="*:ec2_volumes") | eventstats latest(_time) as latest_time | eval latest_time=relative_time(latest_time,"-55m") | dedup id sortby -_time | bin _time span=1mon@mon0 | stats sum(size) by _time, region

This works!

0 Karma

renjith_nair
Legend

@brockwebb, is it worth to check the timestamps in splunk ? i.e. data for previous months are ingested with previous months timestamps or as a batch with single timestamp of current month?

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

adonio
Ultra Champion

can you check if you have older data? you are looking for data 360 days ago, not sure your index supports that retention

0 Karma

brockwebb
Explorer

We have all the data, problem was a pre-defined macro for AWS plugins ... we corrected. Posted answer and description. Thanks for all your help looking into this!!!

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...