Getting Data In

Source rename for IIS logs on default sites

fervin
Path Finder

Hello,

I've noticed an issue with IIS logs that are being written from sites with a default site ID such as (...\logfiles\W3SVC1\ex110826.log). Logs from sites with random site IDs work perfectly fine.

The issue is that logs from these sites all get lumped together when the application builds its summaries.

Changing site IDs across the environment would be a less than ideal approach to fixing this behavior.

Would it be possible to rename these sources at index time and substitute a machine specific element into the source name? It would be perfect if I could transform \logfiles\W3SVC1\ to \logfiles\W3SVC#SERVERNAME or something.

If not, are there any other ways around this? Thanks in advance,

-Frank

1 Solution

sdwilkerson
Contributor

Fervin,

Great question. I have had the same issue, which is, that different physical servers can have the same siteid and therefore all get lumped in with the same source.

I have also implemented your proposed solution at several sites and it works very well.

I created an app called TA-iss to deal with this (TA=Technology-Addon a splunk term for apps that do these sorts of extractions/manipulation). This does index-time manipulation and therefore it should go on whatever system does your parsing (e.g. the forwarder if you are using anything but a Universal Forwarder, or the Indexer if you send directly to the indexer from a Universal Forwarder).

In the props.conf for IIS logs, I have this line:

TRANSFORMS-force_source_for_iis = force_source_for_iis

And the referenced stanza in transforms.conf looks like this:

[force_source_for_iis]
DEST_KEY = MetaData:Source
REGEX = \d{2}\:\d{2}\s+([^\ ]*)\s+([^\ ]*)
FORMAT = source::$2-$1

This makes each source unique to the host-site_id

Best,
Sean

View solution in original post

0 Karma

sdwilkerson
Contributor

Fervin,

Great question. I have had the same issue, which is, that different physical servers can have the same siteid and therefore all get lumped in with the same source.

I have also implemented your proposed solution at several sites and it works very well.

I created an app called TA-iss to deal with this (TA=Technology-Addon a splunk term for apps that do these sorts of extractions/manipulation). This does index-time manipulation and therefore it should go on whatever system does your parsing (e.g. the forwarder if you are using anything but a Universal Forwarder, or the Indexer if you send directly to the indexer from a Universal Forwarder).

In the props.conf for IIS logs, I have this line:

TRANSFORMS-force_source_for_iis = force_source_for_iis

And the referenced stanza in transforms.conf looks like this:

[force_source_for_iis]
DEST_KEY = MetaData:Source
REGEX = \d{2}\:\d{2}\s+([^\ ]*)\s+([^\ ]*)
FORMAT = source::$2-$1

This makes each source unique to the host-site_id

Best,
Sean

0 Karma

araitz
Splunk Employee
Splunk Employee

That seems like a good enhancement request for the app - essentially, facilitate a mapping of the tuple machine-site id-site name.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...