We have multiple web applications that have different information being recorded to make sure the appropriate information is taken from each log. Do we have to assign each type of log a different indexer in inputs.conf while creating new indexers to get exactly what we need from each log?
Don't confuse indexes and indexers. An index is a repository for your data. Indexers are Splunk instances that accept events/logs and store them in indexes.
You do not need separate indexes for each log type. You should, however, specify a sourcetype for each type of log. Sourcetypes tell Splunk how to process the event, such as what the timestamps look like, how the event should be parsed, etc.
Don't confuse indexes and indexers. An index is a repository for your data. Indexers are Splunk instances that accept events/logs and store them in indexes.
You do not need separate indexes for each log type. You should, however, specify a sourcetype for each type of log. Sourcetypes tell Splunk how to process the event, such as what the timestamps look like, how the event should be parsed, etc.
Where can you tell splunk how to process the information based on the sourcetype this was my initial thought, but I was unsure as to how this would occur.
Sourcetypes are defined in the props.conf file. You can also define new sourcetypes while using the Field Extractor.
So if I wanted to create custom sourcetypes, I could add them to the props.conf and edit the inputs.conf and then use the field extractor to edit what key-value pairs they should take from each of these sourcetypes?
Correct. Be sure to modify the config files in the local
directory, not default
.
I had one more additional question regarding this, when I use the field extractor, will this have to be done every time another log is recorded or only when I set it up for that source type.
Once per sourcetype.