All Apps and Add-ons

Stanza [monitor://] not being indexed

corti77
Path Finder

Hi,

My objective is to read the Cluster Shared Volume (CSV) information from a Hyper-V cluster.

I initially achieved it executing a PS1 using the powershell handler [powershell], but the user running UF must have access rights in the cluster so I wanted to do it differently. After some analysis I decided to move the execution of the PS1 to a scheduled task, and simply configure splunk to read the resulting log file of each execution.

The following stanza is the original one running directly the ps1, and it works OK. 

 

#CSV PerfMon Data
[powershell://CSVPerfMetrics]
script = . "$SplunkHome\etc\apps\Splunk_TA_microsoft-hyperv\bin\GetCSVDiskInformation.ps1"
interval = 300
source = microsoft:hyperv:powershell:csvperfmetrics.ps1
sourcetype = microsoft:hyperv:perf:csv
index = hyper-v
disabled = 0

 

Then, I created the scheduled task which creates a different log file on each execution and I tried the stanza bellow without success.

 

[monitor://C:\Scritps\CSV\Outputs\*.log]
sourcetype = microsoft:hyperv:perf:csv
queue = indexQueue 
index = hyper-v
disabled = 0

 

the powershell script

 

$csvdata = Get-ClusterSharedVolume -Cluster XXXXXXXXXX
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
    $csv = [PSCustomObject]@{
        VolumeName = $csvitem.Name; 
        ID = $csvitem.Id; 
        TotalSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.Size); 
        FreeSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
        PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
    }
    $csvs += $csv
}
$timestamp = Get-Date -Format FileDateTime
$csvs | Add-Content -Path "C:\Scritps\CSV\Outputs\CSVRead_$timestamp.log"

 

the content of the log files look like the following:

 

@{VolumeName=Cluster Vol1; ID=4e902f6b-3380-499b-af29-8ff35c02e80d; TotalSpaceKB=2198752718848; FreeSpaceKB=1135195062272; PercentFree=51.62905}
@{VolumeName=Cluster Vol2; ID=002b604e-1671-4054-bed2-c1f8a068b40d; TotalSpaceKB=2198752718848; FreeSpaceKB=789340848128; PercentFree=35.89948}
@{VolumeName=Cluster Vol3; ID=14ce695f-f586-4d65-9e3b-6ab85524fd91; TotalSpaceKB=2198752718848; FreeSpaceKB=1692997120000; PercentFree=76.99807}
@{VolumeName=Cluster Vol4 LinuxRKE; ID=c49e9b9f-d5c1-409c-8ef0-cf8613f81571; TotalSpaceKB=805283295232; FreeSpaceKB=536175865856; PercentFree=66.58227}
@{VolumeName=Cluster Vol5 LinuxSecLAB; ID=f0f7e5e7-1290-4cd4-af66-3c75ffffc1ad; TotalSpaceKB=805283295232; FreeSpaceKB=774000726016; PercentFree=96.11533}

 

any suggestion of what could be happening? any log I could check? So far I didnt find any error message splunkd.log or any other log file, but the indexing simply does not work.

Many thanks

Labels (2)
0 Karma

corti77
Path Finder

Getting closer to the final solution. the first issue was the format of the text file generated. 
After exporting the PS object to JSON, Splunk is capable to ingest it.

$csvdata = Get-ClusterSharedVolume -Cluster LABHQMCLSHV1
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
$timestamp = Get-Date -Format FileDateTime
$csv = [PSCustomObject]@{
#_time = $timestamp;
VolumeName = $csvitem.Name;
ID = $csvitem.Id;
TotalSpaceKB = ($csvitem.SharedVolumeInfo.Partition.Size);
FreeSpaceKB = ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
}

$csv | ConvertTo-Json | Add-Content -Path "C:\Scritps\CSV\Outputs\$timestamp.log"
}

my last issue is how to overwrite the sourcetype of the event. I tried adding the line in inputs.conf but it didnt work. I also create a props.conf with the following line without success

[source::...Outputs\\(.+).log]
sourcetype = microsoft:hyperv:perf:csv

any other idea?

thanks!

 

 

 

 

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!