Combine historical and realtime data from different data types

SplunkHead10 — Mon, 19 Oct 2020 12:24:00 GMT

Hello community,

I used the search to find a possible solution for my problem but without success.
My problem looks the following:

1. I have historical data from a storage sytem which contains the date and the amount used storage (column name historical_gb). This data is in .csv and therefore easy to read.
2. In addition I get the realtime amount of used storage (live_gb) via a REST API. This data is in .json.

My question would be:
How can I combine those two data sources?
The final search should look like:

date	usedStorage
01.01.2020	110 GB	source: historical_gb
19.10.2020	125 GB	source: live_gb

I added the last column just for better understanding. The column is not relevant for the actual search.

So far I have:

index="test" (source = Used_Sotrage_KPI objid = 18392 ) OR (source="historicdata.csv") | rename lastvalue_raw as usedStorage | rename "PhysUsedCapacity_Raw" as usedStorage | timechart span=1d max(usedStorage)

lastvalue_raw = realtime data (source: Used_Storage_KPI)
PhysUsedCapacity_Raw = historical data (source: histrocialdata.csv)

But the second rename overwrites the first rename statement.

Thanks for your help!

Re: Combine historical and realtime data from different data types

SplunkHead10 — Mon, 19 Oct 2020 13:58:59 GMT

This worked for me:

index="test" (source = Used_Sotrage_KPI objid = 18392) OR (source="historicdata.csv") | eval usedStorage = mvappend(lastvalue_raw, 'PhysUsedCapacity_RAW') | timechart span=1d max(usedStorage)

topic Re: Combine historical and realtime data from different data types in Splunk Search

Combine historical and realtime data from different data types

Re: Combine historical and realtime data from different data types