topic Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk? in Getting Data In

How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

akhilchhugani — Tue, 24 May 2016 16:13:42 GMT

We have multiple web applications that have different information being recorded to make sure the appropriate information is taken from each log. Do we have to assign each type of log a different indexer in inputs.conf while creating new indexers to get exactly what we need from each log?

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

richgalloway — Tue, 24 May 2016 17:06:25 GMT

Don't confuse indexes and indexers. An index is a repository for your data. Indexers are Splunk instances that accept events/logs and store them in indexes.

You do not need separate indexes for each log type. You should, however, specify a sourcetype for each type of log. Sourcetypes tell Splunk how to process the event, such as what the timestamps look like, how the event should be parsed, etc.

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

akhilchhugani — Tue, 24 May 2016 17:12:13 GMT

Where can you tell splunk how to process the information based on the sourcetype this was my initial thought, but I was unsure as to how this would occur.

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

richgalloway — Tue, 24 May 2016 17:17:15 GMT

Sourcetypes are defined in the props.conf file. You can also define new sourcetypes while using the Field Extractor.

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

akhilchhugani — Tue, 24 May 2016 17:19:51 GMT

So if I wanted to create custom sourcetypes, I could add them to the props.conf and edit the inputs.conf and then use the field extractor to edit what key-value pairs they should take from each of these sourcetypes?

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

richgalloway — Tue, 24 May 2016 17:25:29 GMT

Correct. Be sure to modify the config files in the local directory, not default.

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

akhilchhugani — Tue, 24 May 2016 20:25:47 GMT

I had one more additional question regarding this, when I use the field extractor, will this have to be done every time another log is recorded or only when I set it up for that source type.

Re: How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

richgalloway — Wed, 25 May 2016 12:15:59 GMT

Once per sourcetype.