topic Field Extraction - restrict extraction to - can this be changed? in Splunk Search

Field Extraction - restrict extraction to - can this be changed?

gerald_huddlest — Mon, 16 Apr 2012 15:43:29 GMT

Field extraction appears to be restricted to Host, Source or sourcetype - I have multiple web servers, and even web services running on the same server - but they all generate different sourcetypes.
Currently I create a Field Extraction per source, but want to know if I can make this more generic and use it across any IIS log. All the logs are given different sourcetype names, to differentiate at search level.
has anyone used the Splunk app for web Intelligence? would this assist?

Re: Field Extraction - restrict extraction to - can this be changed?

kristian_kolb — Mon, 16 Apr 2012 15:59:12 GMT

Uh-oh. Perhaps I'm misunderstanding, but are you setting different sourcetypes for the same type of log file, depending on from which file you're reading?

The best practice is to have the same sourcetype for a certain type of file, regardless of the path/host, e.g. all IIS log files should have sourcetype=iis. Then you can apply all your field extractions on a per sourcetype basis, rather on a per source basis.

Re: Field Extraction - restrict extraction to - can this be changed?

gerald_huddlest — Mon, 16 Apr 2012 16:04:28 GMT

thanks for the response, you are correct.

So on a given host, I have 5 web services running on 5 ports. Each outputs to a separate log directory and I have given them a different sourcetype so that I can easily search against a given source type - would you not recommend this?
Surely my searches are then impacted as I will end up searching against logs for all web services rather than just the specific web service.
Agreed they are the same type of log file.

Re: Field Extraction - restrict extraction to - can this be changed?

kristian_kolb — Mon, 16 Apr 2012 16:29:59 GMT

The way to re-use your field extractions is by applying them at the sourcetype level. This also means that you should set the same sourcetype for all logs of the same type.

While your setup may be better in some cases, this is probably not one of them, since you will have to maintain a bunch of identical field extractions. It may make more sense to change the host value, either through host_segment or host_regex, so that each website gets a unique host name as seen from splunk. You can look this up in the docs for inputs.conf

Or you could limit searches on the source when you want to differentiate between them, e.g.;

sourcetype=iis source=*W3SVC4* | the_rest_of_your_search

Hope this helps,

Kristian