Getting Data In

Organizing Logs

NeoJim
New Member

I'm new and a novice to Splunk although i have installed, setup and played with searches in Splunk in a lab.
My question is if I have servers that are sending logs all from different “environments” (prod, test, dev) what is the best way to organize the logs coming in by environment. I see I can use tags and/or indexes, but which way would make more sense.

Labels (2)
0 Karma

SinghK
Builder

Data retention and access and how sensitive is the data will determine how you will classify and store it. Different approaches for diff organisations. Few of the methods have already been listed by @inventsekar and @isoutamo .

0 Karma

inventsekar
Super Champion

>>> I'm new and a novice to Splunk although i have installed, setup and played with searches in Splunk in a lab.

all the best for your splunk journey!


>>> My question is if I have servers that are sending logs all from different “environments” (prod, test, dev) what is the best way to organize the logs coming in by environment.

Generally we will use index/host/source/sourcetype combinations to identify/group log events

(in bigger environments you will automatically have separate dev / test / prod environments)


>>> I see I can use tags and/or indexes, but which way would make more sense.

Yes, tags are a good way to group logs. more opt way would be source/sourcetype. 

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

You should figure out if there are some regulations which put some requirements for you (e..g separate dev/test and prod access/data). Then there could be some restrictions from your company side which define how to handle data and access to it.

Then in splunk point of view you should thing how long you want/need to store data (clarify it based on that).  Another option is who can access it. This one is the cardinality and amount of ingested data. These are good starting point to divide event to separate indexes. Over that you should/could also use tags, event types etc. to manage events more easily.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...