I installed the Template for Citrix XenApp app yesterday and have a few immediate issues. The first thing is that now all of my searches, regardless of what app, were looking for a "farmHosts" file that doesn't exist and spitting out a mess of errors during the searches. I had to comment out most of the props.conf file to resolve the issues, specifically:
[WMI:Services]
REPORT-DisplayName = extra_service_display_name
LOOKUP-WMI:Services Host Farm Lookup = farmHosts host AS host OUTPUTNEW FarmName AS FarmName
LOOKUP-WMI:Services Service Group Lookup = serviceGroups Name as Name OUTPUTNEW ServiceGroup AS ServiceGroup
[(?::){0}PerfmonMk:*]
LOOKUP-PerfmonMK Host Farm Lookup = farmHosts host AS host OUTPUTNEW FarmName AS FarmName
[(?::){0}WinEventLog:*]
LOOKUP-WinEventLog Host Farm Lookup = farmHosts host AS host OUTPUTNEW FarmName AS FarmName
[(?::){0}xenapp:*:installedsoftware]
LOOKUP-Installed Software Host Farm Lookup = farmHosts host AS host OUTPUTNEW FarmName AS FarmName
Can anyone give me some insight on this "farmHosts" that it is trying to reference that seems to not exist?
The farmHosts lookup is created by a scheduled saved search (that also runs at Splunk startup). The actual file name associated with the farmHosts lookup is "lookup_host_farm.csv" which will be located in $SPLUNK_HOME/etc/apps/TemplateForXenApp/lookups on your Splunk server.
The reason we need this lookup file is to associate data that has no concept of a Citrix XenApp farm (like perfmon, event log, and WMI data) to a Citrix XenApp farm. The farmHosts lookup adds a FarmName field to all of this data so that you can search for things like server performance by farm.
You can manually build/rebuild this file by going to Help -> Rebuild Lookup Files -> Rebuild Host to Farm Lookup File within the Template for Citrix XenApp menu bar. If this does not work, you may not have the appropriate TAs deployed to your XenApp servers.
We upgraded from the legacy Splunk for Citrix XenApp and obviously replaced it with Templates for Citrix XenApp. Was there a change in the TAs from one to the other? Just not sure why I couldn't drop in the upgrade and have things work better. I've had to modify some of the sources for the searches to work in some cases, including the manual build/rebuild. Even after doing that I had still gotten the error messages when running searches (from all apps not just Citrix apps) that it couldn't find "farmHosts" even though there is a file named "lookup_host_farm.csv".
The Template for Citrix XenApp is a separate app with separate TAs that the legacy Splunk for Citrix XenApp app. Therefore, template isn't an "upgrade" so to speak - although the template does incorporate more use cases.
The reason you get errors from other searches is due to the scope of the knowledge objects (searches, transforms, lookups, reports, etc.) in the template. By default, these knowledge objects are set to global. You can change this by either using the Splunk UI (by change the scope of the lookups to app instead of global), or by modifying the local.meta file within the template (make a copy of default.meta and rename to local.meta if it doesn't exist). The reason the knowledge objects are set to global is other apps can use them. Setting knowledge object to global is the default and can be really useful as you perform ad hoc searches.
I found the source of the issue. The "lookup_host_farm.csv" was actually named "lookup_host_farm_staging.csv" and I'm not sure why the difference. When I looked in the props.conf of course I saw the references but it was actually in the transforms.conf where it was pointing to "lookup_host_farm.csv". Once I changed the name I no longer received the errors.
There is a scheduled saved search named "Lookup - Host to Farm" that runs every 24 hours which writes to the _staging file. The search looks like this:
`xd_index` sourcetype=xenapp:*:server | inputlookup lookup_host_farm_staging.csv append=t | stats count by host FarmName | table host FarmName | outputlookup lookup_host_farm_staging.csv | append [ | updatehostfarmlookup ]
The last command in that search "updatehostfarmlookup" copies the _staging file to the real file. The reason it is done this way is you get a file lock error if you try to read from and write to the same file in a single search when Splunk Enterprise is running on a Windows platform.