Getting Data In

Organizing Logs

NeoJim
New Member

I'm new and a novice to Splunk although i have installed, setup and played with searches in Splunk in a lab.
My question is if I have servers that are sending logs all from different “environments” (prod, test, dev) what is the best way to organize the logs coming in by environment. I see I can use tags and/or indexes, but which way would make more sense.

Labels (2)
0 Karma

SinghK
Builder

Data retention and access and how sensitive is the data will determine how you will classify and store it. Different approaches for diff organisations. Few of the methods have already been listed by @inventsekar and @isoutamo .

0 Karma

inventsekar
Ultra Champion

>>> I'm new and a novice to Splunk although i have installed, setup and played with searches in Splunk in a lab.

all the best for your splunk journey!


>>> My question is if I have servers that are sending logs all from different “environments” (prod, test, dev) what is the best way to organize the logs coming in by environment.

Generally we will use index/host/source/sourcetype combinations to identify/group log events

(in bigger environments you will automatically have separate dev / test / prod environments)


>>> I see I can use tags and/or indexes, but which way would make more sense.

Yes, tags are a good way to group logs. more opt way would be source/sourcetype. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

You should figure out if there are some regulations which put some requirements for you (e..g separate dev/test and prod access/data). Then there could be some restrictions from your company side which define how to handle data and access to it.

Then in splunk point of view you should thing how long you want/need to store data (clarify it based on that).  Another option is who can access it. This one is the cardinality and amount of ingested data. These are good starting point to divide event to separate indexes. Over that you should/could also use tags, event types etc. to manage events more easily.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...