It seems as if the Splunk windows Infra app is looking for data that isn't provided by win 2012 R2 hosts when utilising the winhostmon modular input stanza in the splunk addon for windows app.
In the SplunkTAwindows (splunk addon-for windows) there is an input.
Splunk_TA_windows/default/inputs.conf [WinHostMon://OperatingSystem] interval = 600 disabled = 1 type = OperatingSystem index = windows
The events created by this when using a windows 64bit universal forwarder v6.2.1 are similar to
Type=OperatingSystem OS="Microsoft Windows Server 2012 R2 Standard" Architecture="64-bit" Version="6.3.9600" BuildNumber="9600" BuildType="Multiprocessor Free" ServicePack= SerialNumber="xxxxx-70000-00000-xxxxx" ComputerName="xxxxxxxxx" InstallDate="20141216212029.000000+660" LastBootUpTime="20150204160933.812538+660" Locale="0c09" TotalPhysicalMemoryKB="33553908" FreePhysicalMemoryKB="27141004" TotalVirtualMemoryKB="38534644" FreeVirtualMemoryKB="31515452" Status="OK" CodeSet="1252" CountryCode="61" SystemDevice="\Device\HarddiskVolume2" SystemDrive="C:" SystemDirectory="C:\Windows\system32"
Note : that there is no reference to domain anywhere here.
In the splunkappforwindowsinfra/default/savedsearches.conf it references this non-existant field in its lookup builds.
[WinApp_Lookup_Build_Hostmon_Machine - Update - Detail] disabled = 0 is_visible = true action.email.inline = 1 alert.digest_mode = True alert.severity = 1 alert.suppress = 0 alert.track = 0 cron_schedule = 5 * * * * enableSched = 1 dispatch.earliest_time = -80m dispatch.latest_time = now run_on_startup = true search = index=* eventtype="hostmon_windows" Type=OperatingSystem | join host [search eventtype=hostmon_windows Type=Computer earliest=-80m] | stats count by OS, Domain, Architecture, Manufacturer | eval _key = OS . "___" . Domain . "___" . Architecture . "___" . Manufacturer | outputlookup windows_hostmon_machine_details append=true
This results in an empty file so the lookup's do not work in any of the dashboards.
Even when left blank dashboard EXPLICITLY reference the Domain field with search language like " | search Host="" Domain="" OS="*" " as such these fail as there are no Domain fields. Removing the domain limitation will show matching results.
TL;DR , either the Splunk-addon-for-windows is unable to provide a required field for the Splunkappforwindowsinfra app OR the infra app utilises fields that the forwarders can never provide.
I've asked the customer about those addon's and turns out they had always had this installed and it doesn't report domain. The example output in my original post was from that machine for type=operatingsystem.
I've checked all data coming from this particular machine and domain information only exists in the "type" fields :
Active Directory Domain Services
The lookup searches use Type=Computer OR Type=OperatingSystem.