All Apps and Add-ons

How to selectively forward the log files to specific indexes in Splunk?

varunmalhotra
Engager

Is it possible to selectively forward the log files to specific indexes in Splunk.

I want to forward a docker container running 3 services logs to Splunk indexer, the problem is that if I use Docker logging driver, all the data written to STDOUT goes to the same index and data segregation is not possible. Instead of that I've setup forwarder and able to send logs but all are going to the same index, I want to configure splunk forwarder to send specific logs to a specific index.

0 Karma

outcoldman
Communicator

Let me start from the beginning. Running multiple processes in the same container is an anti-pattern. Try to avoid it as much as possible. Kubernetes, for example, have a great solution for your case, where they can deploy two containers in the same Pod and just setup communication between the containers on the same loopback network interface (127.0.0.1), so for the processes, it will look like they are running in the same container. See https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-v... for details.

If you still want to have all three processes in the same container, you have two options to get the logs in different indices:

routing on indexer

If you can identify logs on side of indexer, you can forward logs as you usually do and using transforms.conf on the indexer side route them to specific index, see http://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf

[nginx_route] 
DEST_KEY = _MetaData:Index 
REGEX = nginx .*
FORMAT = index_nginx

avoiding container logs

Another option. You can create a volume for logs, that you will share between your container and forwarder (Splunk Universal Forwarder) or our collector (https://www.outcoldsolutions.com). And in the configuration define to which index you want to forward these logs. In you container you will need to change how you write logs, instead of stdin, write them to the files.

outcoldman
Communicator

It is not clear, are you running one container with 3 processes or 3 different containers?

0 Karma

varunmalhotra
Engager

One Container with three processes.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...