Getting Data In

How can I use my billing info to create a prediction for the future?

RexStout
Explorer

I've asked about this before and now I've re-loaded the raw data without any modifications. It looks like this (without an actual timestamp):

Month,Billing,MsgType,BillSize,Direction
2013-04,BI70276,ORDHDR,5,SENT
2013-04,BI70276,INVFIL,8,RECV
2013-04,BI70276,ORDHDR,5,SENT
2013-04,BI70276,INVFIL,34,RECV
2013-04,BI70276,ORDHDR,20,SENT
2013-04,BI70276,INVFIL,13,RECV
2013-04,BI70276,ORDHDR,7,SENT
2013-04,BI70276,INVFIL,1,RECV
2013-04,BI70276,ORDHDR,1,SENT
2013-04,BI70276,ORDHDR,5,SENT
2013-04,BI70276,INVFIL,4,RECV
2013-04,BI70276,ORDHDR,6,SENT
2013-04,BI70276,INVFIL,9,RECV
2013-04,BI70276,ORDHDR,12,SENT
2013-04,BI70276,INVFIL,178,RECV.................................etc.

I have this data for every CCYY-MM for the last 53 months, c200k events. So, no actual timestamp for each event.
If I use this:

index=IX Billing=BI70400 MsgType=ORDHDR Direction=SENT | stats sum(BillSize) as MonthSize by Month

...I get the column chart that I expect/want.
How can I use this to create a prediction for the future? We've tried a few variations, based on this, but without success.

Thank you.

0 Karma
1 Solution

tmarlette
Motivator

Honestly, the easiest way to do this is to add:
2013-04-{arbitrary day and time} and then just cascade it using excel for the month and THEN load the data. In order to do a 'predict' of any kind, Splunk must have a _time reference, as it has to plot data points into time in the future.

The easiest way to make this happen is to adjust your data to reflect a timestamp.

In the past on daily data I've used midnight, but on intraday data I use hourly increments, and hourly data i'll use minute increments and then load the data into Splunk.

If you're doing monthly predictions, you'll have to do a |timehcart span=1mthen use the predict function, but as long as you're doing monthly increments, you should be able to force a timestamp like I was mentioning. If you need it by the minute or the hour, it will not be as accurate.

View solution in original post

0 Karma

RexStout
Explorer

As I'm getting my data from an Excel spreadsheet, I took your hint and edited the [raw] data so that my CCYY-MM became CCYY-MM-DD HH:MM:SS:XX.

I can now use, virtually, the same search to get what I want.

Thank you,

Richard aka RexStout

0 Karma

RexStout
Explorer

Hmm, I've re-edited the raw data now.....twice.

I suppose I'd have to run the search again for that index...but I'm fairly sure that...

I'd guess it was: dd-mm-ccyy:hh:mm:ss
I used all 12s for the hh:mm:ss so that it slotted into the middle of the day.
The only other option would've been: dd-mm-ccyy.hh:mm:dd

What's the best format for a Splunk timestamp?

Richard aka RexStout

0 Karma

tmarlette
Motivator

Honestly, the easiest way to do this is to add:
2013-04-{arbitrary day and time} and then just cascade it using excel for the month and THEN load the data. In order to do a 'predict' of any kind, Splunk must have a _time reference, as it has to plot data points into time in the future.

The easiest way to make this happen is to adjust your data to reflect a timestamp.

In the past on daily data I've used midnight, but on intraday data I use hourly increments, and hourly data i'll use minute increments and then load the data into Splunk.

If you're doing monthly predictions, you'll have to do a |timehcart span=1mthen use the predict function, but as long as you're doing monthly increments, you should be able to force a timestamp like I was mentioning. If you need it by the minute or the hour, it will not be as accurate.

0 Karma

RexStout
Explorer

Well, what you're suggesting is exactly what I'd tried but never got it to work. All I got was a sum of the events and a prediction of that too. I was convinced that it was all because I tried to shoe-horn in a fake timestamp.

Back to the drawing board!

Thank you,

Richard aka RexStout.

0 Karma

tmarlette
Motivator

What is the format of the timestamp you used? can you copy and paste it here?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...