topic Re: Why is the Splunk_TA_nix hardware sourcetype not automatically extracted? in Getting Data In

Why is the Splunk_TA_nix hardware sourcetype not automatically extracted?

anewell — Tue, 29 Sep 2020 19:04:04 GMT

We are collecting sourcetype=hardware via the Splunk_TA_nix app (v5.2.3), but the data returned isn't being extracted. The ./bin/hardware.sh script is clearly written to produce tabular data, but I seem to be missing a transform that extracts it properly. Does that transform ship in a different app? Am I doing something wrong? A search-time extraction via multikv isn't useful, as the $1::$2 field naming doesn't happen.

In search, each event looks like this:

KEY                   VALUE
CPU_TYPE              Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
CPU_CACHE             12288 KB
CPU_COUNT             4
HARD_DRIVES           sda (Virtual disk) 200 GB;
NIC_TYPE              <notAvailable>
NIC_COUNT             1
MEMORY_REAL           16334412 kB
MEMORY_SWAP           16777208 kB

What I want is MEMORY_REAL="16334412 kB" etc.

Splunk Enterprise 7.0.2, Splunk_ta_nix 5.2.3, mix of CentOS 6.7 & Amazon Linux

Re: Why is the Splunk_TA_nix hardware sourcetype not automatically extracted?

adonio — Thu, 19 Apr 2018 03:06:50 GMT

did you install the TA on the Search Head?

Re: Why is the Splunk_TA_nix hardware sourcetype not automatically extracted?

anewell — Tue, 29 Sep 2020 19:04:51 GMT

Thanks, good question. Yes, Splunk_TA_nix 5.2.3 installed on Seach Head Cluster as well.

I've tried searching the sourcetype directly on the indexer, or from the main SH, or from a different SH w/ the "Splunk App for Unix and Linux" (https://splunkbase.splunk.com/app/273/) installed. In all cases there is no extraction.

I can write the extraction transform myself, but I dislike making local changes to a splunk-provided mainstream TA. I see there is a version 5.2.4 released; perhaps that will help.

Re: Why is the Splunk_TA_nix hardware sourcetype not automatically extracted?

adonio — Fri, 20 Apr 2018 01:04:54 GMT

very odd, i can see extraction with an older version of the TA
is your TAs permissions set to global?

Re: Why is the Splunk_TA_nix hardware sourcetype not automatically extracted?

brian_rampley — Tue, 29 Sep 2020 19:05:11 GMT

The TA should ship with props for the [hardware] sourcetype. I checked mine (Splunk 7.0.2 running on MacOS, Splunk_TA_nix 5.2.4), and mine includes the following extracts and evals in the default props.conf:

EXTRACT-RealMemory = (?i)MEMORY_REAL\s+(?P<RealMemory>[^ ]*)[ ]?
EXTRACT-SwapMemory = (?i)MEMORY_SWAP\s+(?P<SwapMemory>[^ ]*)[ ]?
EXTRACT-Unit = (?i)MEMORY_REAL\s+\d+\s+(?P<Unit>\w+)?
EVAL-RealMemoryMB = case(match(Unit, "kB"), RealMemory*pow(1024,-1), match(Unit, "KB"), RealMemory*pow(1024,-1), match(Unit, "mB"), RealMemory, match(Unit, "MB"), RealMemory, match(Unit, "gB"), RealMemory*pow(1024,1), match(Unit, "GB"), RealMemory*pow(1024,1), match(Unit, "tB"), RealMemory*pow(1024,2), match(Unit, "TB"), RealMemory*pow(1024,2), match(Unit, "pB"), RealMemory*pow(1024,3), match(Unit, "PB"), RealMemory*pow(1024,3), 1==1, "unknown")
EVAL-SwapMemoryMB = case(match(Unit, "kB"), SwapMemory*pow(1024,-1), match(Unit, "KB"), SwapMemory*pow(1024,-1), match(Unit, "mB"), SwapMemory, match(Unit, "MB"), SwapMemory, match(Unit, "gB"), SwapMemory*pow(1024,1), match(Unit, "GB"), SwapMemory*pow(1024,1), match(Unit, "tB"), SwapMemory*pow(1024,2), match(Unit, "TB"), SwapMemory*pow(1024,2), match(Unit, "pB"), SwapMemory*pow(1024,3), match(Unit, "PB"), SwapMemory*pow(1024,3), 1==1, "unknown")
EXTRACT-cpu_cores = (?i)CPU_COUNT\s+(?P<cpu_cores>[^ \n]*)?
EXTRACT-cpu_type = (?i)CPU_TYPE\s+(?P<cpu_type>[^\n]*)?
EVAL-mem = case(match(Unit, "kB"), RealMemory*pow(1024,-1), match(Unit, "KB"), RealMemory*pow(1024,-1), match(Unit, "mB"), RealMemory, match(Unit, "MB"), RealMemory, match(Unit, "gB"), RealMemory*pow(1024,1), match(Unit, "GB"), RealMemory*pow(1024,1), match(Unit, "tB"), RealMemory*pow(1024,2), match(Unit, "TB"), RealMemory*pow(1024,2), match(Unit, "pB"), RealMemory*pow(1024,3), match(Unit, "PB"), RealMemory*pow(1024,3), 1==1, "unknown")

When I check the data in splunk, I have the following fields that match up to these props: RealMemory, RealMemoryMB, SwapMemory, SwapMemoryMB, cpu_cores, cpu_type, mem

If the add-on is installed on your search heads and indexers, you should get the same field extractions.