All Apps and Add-ons

Windows TA - MultiKV fields

anirban_td
Explorer

Hi all, 

I want to know how splunk extracts fields from TA_windows inputs when mode=multikv 

The _raw event does not seem to have any sort of field indicator (as compared to events from TA_nix which has headers) 

As an example: 

Splunk_TA_windows/local/inputs.conf

 

 

[perfmon://Network-Bytes]
disabled = false
counters = Bytes Total/sec; Bytes Received/sec; Bytes Sent/sec; 
interval = 60
mode = multikv
index = perfmon
useEnglishOnly = true
object = Network Interface
sourcetype = PerfmonMk:Network

 

 

gives _raw events as seen indexed in Splunk: 

 

 

vmxnet3_Ethernet_Adapter 19069.926362422757 11044.290764991998 8025.635597430761
vmxnet3_Ethernet_Adapter 26173.569591676503 15701.614528029395 10471.95506364711
vmxnet3_Ethernet_Adapter 28654.246470518276 17482.977608482255 11171.268862036022

 

 

From this output, splunk magically extracts fields like: 

 

 

Bytes_Received/sec
Bytes_Sent/sec
Bytes_Total/sec
instance
category
collection

 

 

I checked the TA_windows configs and ran btool, but could not trace configs other than some standard PerfmonMk:<object> stanzas in Splunk_TA_windows/default/props.conf which contain only FIELDALIAS settings

What am I missing?
How does splunk know which field is which? 
How does it even get values for category & collection when those values are not even present in the _raw? 

 

Spoiler

Further comparison, TA_nix add-on does this in a much more legible manner (which can be easily understood and played around with) like: 

Name rxPackets_PS txPackets_PS rxKB_PS txKB_PS
eth0 1024.00 1972.50 1415.04 674.94
​
 

 

Additional: 

Spoiler
I want to convert the PerfmonMk events to metrics, has anyone attempted that? 

 

 

Labels (4)

SinghK
Builder

index time field extractions and mappings are all done in props and transforms.conf.

SinghK
Builder

i see you have customized inputs :

this is thedefault one

## Network
[perfmon://Network]
counters = Bytes Total/sec; Packets/sec; Packets Received/sec; Packets Sent/sec; Current Bandwidth; Bytes Received/sec; Packets Received Unicast/sec; Packets Received Non-Unicast/sec; Packets Received Discarded; Packets Received Errors; Packets Received Unknown; Bytes Sent/sec; Packets Sent Unicast/sec; Packets Sent Non-Unicast/sec; Packets Outbound Discarded; Packets Outbound Errors; Output Queue Length; Offloaded Connections; TCP Active RSC Connections; TCP RSC Coalesced Packets/sec; TCP RSC Exceptions/sec; TCP RSC Average Packet Size
disabled = 1
instances = *
interval = 10
mode = multikv
object = Network Interface
useEnglishOnly=true

 

you have customized sourcetype. I will not do that, as there is a lot more working on standard sourcetype in a addon behind the scenes during indexing time.

anirban_td
Explorer

@SinghK wrote:

you have customized sourcetype. I will not do that, as there is a lot more working on standard sourcetype in a addon behind the scenes during indexing time.


i have explicitly mentioned the sourcetype to use in the input, but i have not customised the sourcetype definition

regardless, my question is specifically on the 'behind the scenes' processing that goes on for mode=multikv 

please see my reply to @inventsekar 

PS: if you have any standard best practices for defining TA_windows inputs, feel free to share them.. 

0 Karma

inventsekar
Super Champion

Hi @anirban_td .. i am not much sure of WindowsTA, but, one thing for sure - the windows logs are pretty much formatted.

so, headers may not be needed at all. hope you understand my view, thanks. 

>>> Happy Splunking !
0 Karma

anirban_td
Explorer

again, taking the example of TA_nix bandwidth.sh event: 

 

Name rxPackets_PS txPackets_PS rxKB_PS txKB_PS
eth0 1024.00 1972.50 1415.04 674.94

 

one can easily recognize (and setup extraction mechanisms for) the fields because of the header row... 
however, if the header row is not there, how do you do it? 

-----------------------------------------------------------------------------------------------

i agree the multikv events are well formatted

but i still do not understand how splunk:

  • recognizes the fields correctly in absence of a header row
  • get values for category & collection when those values are not present in the _raw event

the only logical explanation that i can arrive at is: the header row (or something similar, which aids splunk in identifying the fields) is generated at the UF level; but once the event reaches the indexer tier, it is discarded after field extraction, to save license cost & disk space.. 

i want to know :

  • if this assumption is correct
  • the config that processes the PerfmonMk:<> sourcetypes

----------------------------------------------------------------------------

i am sure i am missing SOMETHING here.. 

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...