All Apps and Add-ons

Stanza [monitor://] not being indexed

corti77
Communicator

Hi,

My objective is to read the Cluster Shared Volume (CSV) information from a Hyper-V cluster.

I initially achieved it executing a PS1 using the powershell handler [powershell], but the user running UF must have access rights in the cluster so I wanted to do it differently. After some analysis I decided to move the execution of the PS1 to a scheduled task, and simply configure splunk to read the resulting log file of each execution.

The following stanza is the original one running directly the ps1, and it works OK. 

 

#CSV PerfMon Data
[powershell://CSVPerfMetrics]
script = . "$SplunkHome\etc\apps\Splunk_TA_microsoft-hyperv\bin\GetCSVDiskInformation.ps1"
interval = 300
source = microsoft:hyperv:powershell:csvperfmetrics.ps1
sourcetype = microsoft:hyperv:perf:csv
index = hyper-v
disabled = 0

 

Then, I created the scheduled task which creates a different log file on each execution and I tried the stanza bellow without success.

 

[monitor://C:\Scritps\CSV\Outputs\*.log]
sourcetype = microsoft:hyperv:perf:csv
queue = indexQueue 
index = hyper-v
disabled = 0

 

the powershell script

 

$csvdata = Get-ClusterSharedVolume -Cluster XXXXXXXXXX
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
    $csv = [PSCustomObject]@{
        VolumeName = $csvitem.Name; 
        ID = $csvitem.Id; 
        TotalSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.Size); 
        FreeSpaceKB =  ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
        PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
    }
    $csvs += $csv
}
$timestamp = Get-Date -Format FileDateTime
$csvs | Add-Content -Path "C:\Scritps\CSV\Outputs\CSVRead_$timestamp.log"

 

the content of the log files look like the following:

 

@{VolumeName=Cluster Vol1; ID=4e902f6b-3380-499b-af29-8ff35c02e80d; TotalSpaceKB=2198752718848; FreeSpaceKB=1135195062272; PercentFree=51.62905}
@{VolumeName=Cluster Vol2; ID=002b604e-1671-4054-bed2-c1f8a068b40d; TotalSpaceKB=2198752718848; FreeSpaceKB=789340848128; PercentFree=35.89948}
@{VolumeName=Cluster Vol3; ID=14ce695f-f586-4d65-9e3b-6ab85524fd91; TotalSpaceKB=2198752718848; FreeSpaceKB=1692997120000; PercentFree=76.99807}
@{VolumeName=Cluster Vol4 LinuxRKE; ID=c49e9b9f-d5c1-409c-8ef0-cf8613f81571; TotalSpaceKB=805283295232; FreeSpaceKB=536175865856; PercentFree=66.58227}
@{VolumeName=Cluster Vol5 LinuxSecLAB; ID=f0f7e5e7-1290-4cd4-af66-3c75ffffc1ad; TotalSpaceKB=805283295232; FreeSpaceKB=774000726016; PercentFree=96.11533}

 

any suggestion of what could be happening? any log I could check? So far I didnt find any error message splunkd.log or any other log file, but the indexing simply does not work.

Many thanks

Labels (2)
0 Karma

corti77
Communicator

Getting closer to the final solution. the first issue was the format of the text file generated. 
After exporting the PS object to JSON, Splunk is capable to ingest it.

$csvdata = Get-ClusterSharedVolume -Cluster LABHQMCLSHV1
$csvs = @()
foreach ( $csvitem in ($csvdata))
{
$timestamp = Get-Date -Format FileDateTime
$csv = [PSCustomObject]@{
#_time = $timestamp;
VolumeName = $csvitem.Name;
ID = $csvitem.Id;
TotalSpaceKB = ($csvitem.SharedVolumeInfo.Partition.Size);
FreeSpaceKB = ($csvitem.SharedVolumeInfo.Partition.FreeSpace);
PercentFree = $csvitem.SharedVolumeInfo.Partition.PercentFree;
}

$csv | ConvertTo-Json | Add-Content -Path "C:\Scritps\CSV\Outputs\$timestamp.log"
}

my last issue is how to overwrite the sourcetype of the event. I tried adding the line in inputs.conf but it didnt work. I also create a props.conf with the following line without success

[source::...Outputs\\(.+).log]
sourcetype = microsoft:hyperv:perf:csv

any other idea?

thanks!

 

 

 

 

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...