Veeam Backup TA

[archspangler]

This application would be much better if it didn't dump the entire Job Backup Session history every time the scheduled tasks runs. Our JOBTSK* files have about 180K events in each file. Of the 180K events there are only a handful of new lines each time it runs; why re-index a new source for this sourcetype every schedule task run for the addition of a few line? This actually applies to all the sourcetype in this TA. This just doesn't seem very efficient and friendly to a Splunk license model.

[srobinsonxtl]

christopherHall, you will need to do the following to get the last hour of jobs.

$date1 = $(get-date).AddHours(-1)

Get-VBRBackupSession | Get-VBRTaskSession | Select @{N="#Date";E={$.Info.QueuedTime}}, @{N="Id";E={$.Id}},@{N="Name";E={$.Name}}, @{N="JobName";E={$.JobName}}, @{N="JobSessId";E={$.JobSessId}}, @{N="Status";E={$.Status}}, @{N="Reason";E={$_.Info.Reason}} ##| Export-csv -Path C:\Temp\splunk\"JOBTSK"$Epoch.csv -NoTypeInformation

Thanks,

Stephen Robinson

[lqiao2]

Hi Stephen, it sounds to me that you made it work in your environment? Not sure if you are actively use it. If so, what would be your suggestions to make this app work? I want to use it to monitor our Veeam Backup status. Thanks.

[srobinsonxtl]

I have been using it but currently not as much because there were somethings I wanted to modify for my usage and just didn't have time to do that. I will be happy to discuss it with you. If you want to connect, you can call me or I can call you. If there is a way to send a private message, i will provide you my contact information.

Thanks,

Stephen Robinson

[lqiao2]

Thanks. Leveraging Splunk Support to exchange our contact information. Hang on there!

[christopherHal1]

I get the update to the date1 variable but i dont see that variable being called at all?? i see the select@{N="#Date" But again i dont see where its being called.

[srobinsonxtl]

I am sorry, please see the following, I guess it was too late when I sent this to you (Ha Ha Ha)

$date1 = $(get-date).AddHours(-1)

Get-VBRBackupSession | where {$.EndTime -gt $date1} | Get-VBRTaskSession | Select @{N="#Date";E={$.Info.QueuedTime}}, @{N="Id";E={$.Id}},@{N="Name";E={$.Name}}, @{N="JobName";E={$.JobName}}, @{N="JobSessId";E={$.JobSessId}}, @{N="Status";E={$.Status}}, @{N="Reason";E={$.Info.Reason}} ##| Export-csv -Path C:\Temp\splunk\"JOBTSK"$Epoch.csv -NoTypeInformation

[christopherHal1]

I typed up a nice long response and then clicked cancel .... AWESOME.

Anyways I get log data but its all timestamped exactly with what ever the $date1 variable is.
Script Example.

Add-PSSnapin VeeamPSSnapin
$date1 = (get-date).AddHours(-1)
$date2 = Get-Date

$Epoch = (New-TimeSpan -Start $date1 -End $date2).TotalSeconds
Get-VBRViProxy -Name * | Select @{N="#Date";E={$date1}}, @{N="Name";E={$.Name}}, @{N="Host";E={$.Host}}, @{N="Type";E={$.Type}}, @{N="IsDisabled";E={$.IsDisabled}},@{N="Description";E={$_.Description}} | Export-csv -Path C:\Temp\splunk\"BKPPXY"$Epoch.csv -NoTypeInformation

EXAMPLE Variable:
PS C:\Windows\system32> $date1
Friday, October 5, 2018 1:00:12 PM

EXAMPLE Out PUT:
"#Date","Name","Host","Type","IsDisabled","Description"
"10/5/2018 1:00:12 PM","SERVERNAME","Veeam.Backup.Core.Common.CHost","Vi","False","Proxy and Repository Server on UCS Chassis located in LOCATION."

I get about 20 items back with the same exact timestamp which is in line with the VAR. I wanted to get the range of events rather than just the one timestamp. I assum its something wrong in the loggic.

[jnoga]

Thanks for the info on the TA. I will update the source scripts in the TA to reflect the request I’ll put a check in the script to prevent the entire read of the job history. In the meantime you can increase duration manually of the scheduled job to prevent it from running excessively.

[srobinsonxtl]

Hello,

I am at a point now that I can help work on this. Would you mind having a call to discuss the application and what you would like to do going forward. I know powershell pretty well, so I am sure I can help determine the best way to get the data and how to avoid the duplicate data.

Thanks,

Stephen Robinson

[DBuhler]

Have you updated the app? I only can see it on version 1, and was thinking to start using it to monitor my veeam.

[srobinsonxtl]

I am not the curator of the app but bgstein is. As far as I know, there is currently one app. I have talked to bgstein about improving the application but we haven't discuss anything as of yet.

Thanks,

Stephen Robinson

[christopherHal1]

Guys, im just manually running one Get from thepowershell to see the results. More specifically:

Get-VBRBackupSession -Name * | Get-VBRTaskSession | Select @{N="#Date";E={$.Info.QueuedTime}}, @{N="Id";E={$.Id}},@{N="Name";E={$.Name}}, @{N="JobName";E={$.JobName}}, @{N="JobSessId";E={$.JobSessId}}, @{N="Status";E={$.Status}}, @{N="Reason";E={$_.Info.Reason}} ##| Export-csv -Path C:\Temp\splunk\"JOBTSK"$Epoch.csv -NoTypeInformation

I've only been able to get it to return results once i assume its because of the time variables. I tried updating

$date1 = (get-date).AddHours(-1)

In hopes that i only gather the data for the last hour.
Is this how you guys are approaching this? It is my first attempt at playing with the powershell for veeam.

[srobinsonxtl]

christopherHall, you will need to do the following to get the last hour of jobs.

$date1 = $(get-date).AddHours(-1)

Get-VBRBackupSession | Get-VBRTaskSession | Select @{N="#Date";E={$.Info.QueuedTime}}, @{N="Id";E={$.Id}},@{N="Name";E={$.Name}}, @{N="JobName";E={$.JobName}}, @{N="JobSessId";E={$.JobSessId}}, @{N="Status";E={$.Status}}, @{N="Reason";E={$_.Info.Reason}} ##| Export-csv -Path C:\Temp\splunk\"JOBTSK"$Epoch.csv -NoTypeInformation

Thanks,

Stephen Robinson

[srobinsonxtl]

Jnoga,

I just saw your application and loved what you did with it. I am wondering if you are still actively supporting this product? I found a couple of things that didn't work and made the minor changes to resolve this issue. I am interesting in furthering this application as more data is provided through Veeam.

Please comment back and let me know how I can contact you.

Thanks,

Stephen Robinson

[bgstein]

Would you be willing to post you modifications or recommendations?

[srobinsonxtl]

I would love to, I really don't remember all I did, but I will see if I can find some notes. The only thing I remember at this time is all the duplicate data.

I am out of town starting tomorrow, so will not be able to get back to it until next week. I also have some other application I would like develop, and maybe you would be interested in collaborating.

Thanks,

Stephen Robinson

[bgstein]

I'd appreciate your taking the time.

To document the problem, the app uses a powershell script to pull information from Veeam. Veeam provides a powershell snapin VeeamPSSnapin. The script creates csv files that are picked up by the splunk forwarder. The app also tried to create a scheduled task to run the powershell script. On my box I had to manually create the task.

First question I'm working on is how often should the task run? I modified the script to set date1 as follows and run the script currently daily:
$date1 = (Get-Date).AddDays(-1)

Second issue I had was the script not connecting, so I added before the get queries:
Connect-VBRServer

Current problem I have is with the line starting: Get-VBRJob -Name * | Select @{N="#Date";E={$_.FindLastSession().Progress.StartTime}} This is returning no value which throws a bunch of stuff off. I've set it to date2 and am now trying to understand enough to figure out why the original code doesn't work.

The duplicate data you mention is I think from the line creating JOBTSK with Get-VBRBackupSession.

[srobinsonxtl]

Hello,

I am at a point now that I can help work on this. Would you mind having a call to discuss the application and what you would like to do going forward. I know powershell pretty well, so I am sure I can help determine the best way to get the data and how to avoid the duplicate data.

Thanks,

Stephen Robinson

[dominiquevocat]

oh wait, i downloaded the app and if i see it correctly then you only need to do this for the start date
line 6:

$date1 = Get-Date.AddHours(-1)

if it runs hourly... no idea what the schedule is though ...

[dominiquevocat]

hi, did you ever get around to do it? The app is still version 1.0 so i guess it has not yet been updated?