Deployment Architecture

How to distinguish applications log by environment?

GaetanVP
Contributor

Hello Splunkers,

Here is my use-case : I am monitoring apache logs on 3 different VMs, one VM for each env : dev, uat, prod

I do not see the point to create a specific index for each env (no security / restrictions needed). But I still want to be able to distinguish the logs by environment. 

What would be the best practice to do that ? Create a tag / event type ? for each host from where the logs are coming ?

Regards,
GaetanVP

Labels (1)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

It depends on your case.

If you don't need to restrict access to particular dev (like you have a team which should see logs from dev but not from prod) and don't need to apply different retention policies you generally don't need to split the events into different indexes. (OK, if your dev or uat generated way way less events than prod, that could also be the case for dedicating a separate index for it).

Assuming that you don't split it into different indexes, question is whether your data is distinguishable by any part of data/metadata. If - for example - your host field is different, you're all set. You know how to search for data from prod (by limiting your search with "host IN (hosta,hostb, hostc)". To make search more convenient you can create an eventtype or alias (typically alias is often used to externalize a configurable part from searches so that you can make many static searches - for example in a dashboard - and you can easily change set of data involved used by just redefining the macro).

If you don't have any metadata field which can help you tell the source environment, you have to generate one by means of additional indexed field.

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

It depends on your case.

If you don't need to restrict access to particular dev (like you have a team which should see logs from dev but not from prod) and don't need to apply different retention policies you generally don't need to split the events into different indexes. (OK, if your dev or uat generated way way less events than prod, that could also be the case for dedicating a separate index for it).

Assuming that you don't split it into different indexes, question is whether your data is distinguishable by any part of data/metadata. If - for example - your host field is different, you're all set. You know how to search for data from prod (by limiting your search with "host IN (hosta,hostb, hostc)". To make search more convenient you can create an eventtype or alias (typically alias is often used to externalize a configurable part from searches so that you can make many static searches - for example in a dashboard - and you can easily change set of data involved used by just redefining the macro).

If you don't have any metadata field which can help you tell the source environment, you have to generate one by means of additional indexed field.

scelikok
SplunkTrust
SplunkTrust

Hi @GaetanVP,

Actually, you do not need anything specific to distinguish. They are already different hosts. You can easily search using the host like below;

index=apache host=dev

  Or you can check host field to distinguish the environment

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

tej57
Contributor

Hello @GaetanVP,

You definitely do not need to create 3 different indexes for each environment. However based on the host, you can route them to a specific sourcetype by using below props.conf.

props.conf

[host::<<dev_host>>]
sourcetype = <<orig_sourcetype:dev>>

[host::<<uat_host>>]
sourcetype = <<orig_sourcetype:uat>>

[host::<<prod_host>>]
sourcetype = <<orig_sourcetype:prod>>

This will ingest all the data into one index and whenever you wish to search the logs for specific environment, you can filter the logs based on the sourcetype. Additionally, if you expand the number of hosts in any environment, you can provide the regex pattern for hostnames to route them to env specific sourcetype.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

No!

Don't touch the sourcetype!

Sourcetype is the most basic level of saying "this format of events contains this and that" so if you change the sourcetype you'd have to adjust a whole bunch of other configuration items. So no, sourcetype is not the way to go.

Not to mention that your syntax won't work anyway.

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...