All Apps and Add-ons

Stanza [monitor://] not being indexed

corti77
Communicator

Hi,

My objective is to read the Cluster Shared Volume (CSV) information from a Hyper-V cluster.

I initially achieved it executing a PS1 using the powershell handler [powershell], but the user running UF must have access rights in the cluster so I wanted to do it differently. After some analysis I decided to move the execution of the PS1 to a scheduled task, and simply configure splunk to read the resulting log file of each execution.

The following stanza is the original one running directly the ps1, and it works OK. 

 

#CSV PerfMon Data
[powershell://CSVPerfMetrics]
script = . "$SplunkHome\etc\apps\Splunk_TA_microsoft-hyperv\bin\GetCSVDiskInformation.ps1"
interval = 300
source = microsoft:hyperv:powershell:csvperfmetrics.ps1
sourcetype = microsoft:hyperv:perf:csv
index = hyper-v
disabled = 0

 

Then, I created the scheduled task which creates a different log file on each execution and I tried the stanza bellow without success.

 

[monitor://C:\Scritps\CSV\Outputs\*.log]
sourcetype = microsoft:hyperv:perf:csv
queue = indexQueue 
index = hyper-v
disabled = 0

 

the powershell script

 

$csvdata = Get-ClusterSharedVolume -Cluster XXXXXXXXXX
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
    $csv = [PSCustomObject]@{
        VolumeName = $csvitem.Name; 
        ID = $csvitem.Id; 
        TotalSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.Size); 
        FreeSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
        PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
    }
    $csvs += $csv
}
$timestamp = Get-Date -Format FileDateTime
$csvs | Add-Content -Path "C:\Scritps\CSV\Outputs\CSVRead_$timestamp.log"

 

the content of the log files look like the following:

 

@{VolumeName=Cluster Vol1; ID=4e902f6b-3380-499b-af29-8ff35c02e80d; TotalSpaceKB=2198752718848; FreeSpaceKB=1135195062272; PercentFree=51.62905}
@{VolumeName=Cluster Vol2; ID=002b604e-1671-4054-bed2-c1f8a068b40d; TotalSpaceKB=2198752718848; FreeSpaceKB=789340848128; PercentFree=35.89948}
@{VolumeName=Cluster Vol3; ID=14ce695f-f586-4d65-9e3b-6ab85524fd91; TotalSpaceKB=2198752718848; FreeSpaceKB=1692997120000; PercentFree=76.99807}
@{VolumeName=Cluster Vol4 LinuxRKE; ID=c49e9b9f-d5c1-409c-8ef0-cf8613f81571; TotalSpaceKB=805283295232; FreeSpaceKB=536175865856; PercentFree=66.58227}
@{VolumeName=Cluster Vol5 LinuxSecLAB; ID=f0f7e5e7-1290-4cd4-af66-3c75ffffc1ad; TotalSpaceKB=805283295232; FreeSpaceKB=774000726016; PercentFree=96.11533}

 

any suggestion of what could be happening? any log I could check? So far I didnt find any error message splunkd.log or any other log file, but the indexing simply does not work.

Many thanks

Labels (2)
0 Karma

corti77
Communicator

Getting closer to the final solution. the first issue was the format of the text file generated. 
After exporting the PS object to JSON, Splunk is capable to ingest it.

$csvdata = Get-ClusterSharedVolume -Cluster LABHQMCLSHV1
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
$timestamp = Get-Date -Format FileDateTime
$csv = [PSCustomObject]@{
#_time = $timestamp;
VolumeName = $csvitem.Name;
ID = $csvitem.Id;
TotalSpaceKB = ($csvitem.SharedVolumeInfo.Partition.Size);
FreeSpaceKB = ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
}

$csv | ConvertTo-Json | Add-Content -Path "C:\Scritps\CSV\Outputs\$timestamp.log"
}

my last issue is how to overwrite the sourcetype of the event. I tried adding the line in inputs.conf but it didnt work. I also create a props.conf with the following line without success

[source::...Outputs\\(.+).log]
sourcetype = microsoft:hyperv:perf:csv

any other idea?

thanks!

 

 

 

 

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...