All Apps and Add-ons

splunk-connect-for-kubernetes - /var/log/containers/ log unreadable. It is excluded and would be examined next time.

xindeNokia
Path Finder

After deploy splunk-connect-for-k8s 1.3, I saw lots of warning msg from splunk-splunk-kubernetes-logging:

2020-02-19 02:50:09 +0000 [warn]: #0 [containers.log] /var/log/containers/splunk-splunk-kubernetes-logging-4wqnj_default_splunk-fluentd-k8s-logs-e403ac1d989b252566536f844f3817ac9334ac1dbc80bbfeda04fb063dac65a8.log unreadable. It is excluded and would be examined next time.

it seems all of the pods logs are skipped and unreadable, I can find them under /var/log/containers, but they are softlinks.
Not sure if this caused any issues. K8s cluster is deployed by kubespray.

lrwxrwxrwx 1 root root 127 Feb 18 21:47 splunk-splunk-kubernetes-logging-4wqnj_default_splunk-fluentd-k8s-logs-e403ac1d989b252566536f844f3817ac9334ac1dbc80bbfeda04fb063dac65a8.log -> /var/log/pods/default_splunk-splunk-kubernetes-logging-4wqnj_56c07fb2-d86d-411b-92a4-7c214919a33d/splunk-fluentd-k8s-logs/0.log

Wonder if anyone has seen this before and maybe I misconfigured something or there is any solutions for this...

Thanks in advance!

0 Karma
1 Solution

xindeNokia
Path Finder

Found the solution for my question -

./splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging/values.yaml: path: /var/log/containers/.log
Changed to:
path: /var/log/pods/
.log works to me.

View solution in original post

0 Karma

xindeNokia
Path Finder

Found the solution for my question -

./splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging/values.yaml: path: /var/log/containers/.log
Changed to:
path: /var/log/pods/
.log works to me.

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...