Getting Data In

How can we make sure the appropriate key-value pairs from different web application logs are indexed in Splunk?

akhilchhugani
New Member

We have multiple web applications that have different information being recorded to make sure the appropriate information is taken from each log. Do we have to assign each type of log a different indexer in inputs.conf while creating new indexers to get exactly what we need from each log?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Don't confuse indexes and indexers. An index is a repository for your data. Indexers are Splunk instances that accept events/logs and store them in indexes.

You do not need separate indexes for each log type. You should, however, specify a sourcetype for each type of log. Sourcetypes tell Splunk how to process the event, such as what the timestamps look like, how the event should be parsed, etc.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Don't confuse indexes and indexers. An index is a repository for your data. Indexers are Splunk instances that accept events/logs and store them in indexes.

You do not need separate indexes for each log type. You should, however, specify a sourcetype for each type of log. Sourcetypes tell Splunk how to process the event, such as what the timestamps look like, how the event should be parsed, etc.

---
If this reply helps you, Karma would be appreciated.

akhilchhugani
New Member

Where can you tell splunk how to process the information based on the sourcetype this was my initial thought, but I was unsure as to how this would occur.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Sourcetypes are defined in the props.conf file. You can also define new sourcetypes while using the Field Extractor.

---
If this reply helps you, Karma would be appreciated.
0 Karma

akhilchhugani
New Member

So if I wanted to create custom sourcetypes, I could add them to the props.conf and edit the inputs.conf and then use the field extractor to edit what key-value pairs they should take from each of these sourcetypes?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Correct. Be sure to modify the config files in the local directory, not default.

---
If this reply helps you, Karma would be appreciated.
0 Karma

akhilchhugani
New Member

I had one more additional question regarding this, when I use the field extractor, will this have to be done every time another log is recorded or only when I set it up for that source type.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Once per sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...