ITSI, IIS logs and splitting by sitename

[eddieddieddie]

Hi

I'm trying to use ITSI to use KPI's from IIS servers. The setup of the IIS web servers is they host several different sites and in ITSI I want to break this out into different services.

Splunk is ingesting the IIS logs successfully - the data includes the hostname of the server it's running on and the site name.

In ITIS I've setup a new service. For the entities of this service I've made a rule to match both the alias fields 'host' and 'site' (and made sure both these fields are set on the Entity record in ITSI).

Then I setup a new KPI using a base search to count the number of 5xx errors. This is set to split by the field 'host' - the website is hosted from two separate machines. Then filtered by service entities in field 'site'.

This seemed to work until I started creating other services for other websites. I wanted to also monitor the non-production version of this website. So I created a service as above but using the non-prod host names, however the site name is the same. The result of this was really weird: the KPI then listed the production and non-production servers in the entity list for this service (though they are not in the Entities list for that service).

ITSI also started giving warnings of duplicate alias's assigned to entities. At this point I thought maybe I was defining the 'site' on the entity in the wrong way. So I moved site from being an alias to 'Info'. But unfortunately ITSI doesn't seem to be able to filter by the info field.

I guess the issue I'm facing here is I need a way to filter an entity by two fields - the hostname of the server(s) it's on and the 'site' name in IIS. How is this archived?

Thanks, Eddie

[eduncan]

You can do this filtering but the alias field must be unique or you will have duplicate entities.

Do you have access to a lookup where are you could list out the service name and which servers support each?  You could then do a look up and an entity import search that pose in a service name and then in your entity filter you just use that.

If you’d like to have a  session and I could show you let me know a good way to send you my info.

[eddieddieddie]

Hi and thanks for the quick response.

Yes - could you give more details on the lookup and entity import search you mention in your reply?

Here is a list of servers, IIS site names, and Service Name across both environments. The whole setup is bigger than this but hopefully gives you an idea. Note: to further confuse things the IIS site name is reused on different servers for different purposes.

Environment	Service Name	IIS Site Name	Server Host Name
Non-production	User Interface	UserInterface	QAUI11
			QAUI12
	Data Processing	DataProcessing	QAAPP11
			QAAPP12
	Image Processing	DataProcessing	QAAPP15
			QAAPP16
	Reporting	ReportingInterface	QAUI11
			QAUI12
Production	User Interface	UserInterface	PRDUI11
			PRDUI12
	Data Processing	DataProcessing	PRDAPP11
			PRDAPP12
	Image Processing	DataProcessing	PRDAPP15
			PRDAPP16
	Reporting	ReportingInterface	PRDUI11
			PRDUI12

Ideally I would like to be able to create a service in ITIS for each item in the 'Service Name' column which then has KPIs monitoring the performance of IIS for only the servers and IIS sites that serve that service.

[eduncan]

Ok so you need to first, make sure you are not using an alias field in any entity that is not unique.  Every entity alias must be a unique name just like a unique entity name.  Also make sure those other fields in your lookup table are entity information fields in your entity. When you have it set, inside of the service, under the entity tab, change the field name to 'Info' and then choose the service name value and on the right side enter what service name you want to include.  This allows you to filter by that information field and not a field in the raw data.  You just need to be sure that inside of your KPI search, you do the exact same lookup in your search command so that the field actually exists in the search returned.

[eddieddieddie]

Thanks for this addition information. As you recommended I have changed the entity removing the ‘site’ field from the alias section and recreating it in the ‘Info fields’ section – as below:

I then opened the Service and from the Entities Tab, removed ‘site’ as an alias and recreated it as an Info field – so it looks like the following. It does successfully match the correct entity too.

In the KPI the ‘entity filter field’ is set to ‘site’.

But now the KPI does not work! In the data I can manually search and find ‘[rest of the base search] site=UserInterface’ via search. If I expand the Generated Search for the KPI, open it in search, and reduce the search just back to the base search and the rest lookup I get nothing. So it seems that the generate_entity_filter function isn’t picking up the field 'site' since I changed it to ‘info’. What am I doing wrong here?