All Apps and Add-ons

splunk-connect-for-kubernetes - /var/log/containers/ log unreadable. It is excluded and would be examined next time.

xindeNokia
Path Finder

After deploy splunk-connect-for-k8s 1.3, I saw lots of warning msg from splunk-splunk-kubernetes-logging:

2020-02-19 02:50:09 +0000 [warn]: #0 [containers.log] /var/log/containers/splunk-splunk-kubernetes-logging-4wqnj_default_splunk-fluentd-k8s-logs-e403ac1d989b252566536f844f3817ac9334ac1dbc80bbfeda04fb063dac65a8.log unreadable. It is excluded and would be examined next time.

it seems all of the pods logs are skipped and unreadable, I can find them under /var/log/containers, but they are softlinks.
Not sure if this caused any issues. K8s cluster is deployed by kubespray.

lrwxrwxrwx 1 root root 127 Feb 18 21:47 splunk-splunk-kubernetes-logging-4wqnj_default_splunk-fluentd-k8s-logs-e403ac1d989b252566536f844f3817ac9334ac1dbc80bbfeda04fb063dac65a8.log -> /var/log/pods/default_splunk-splunk-kubernetes-logging-4wqnj_56c07fb2-d86d-411b-92a4-7c214919a33d/splunk-fluentd-k8s-logs/0.log

Wonder if anyone has seen this before and maybe I misconfigured something or there is any solutions for this...

Thanks in advance!

0 Karma
1 Solution

xindeNokia
Path Finder

Found the solution for my question -

./splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging/values.yaml: path: /var/log/containers/.log
Changed to:
path: /var/log/pods/
.log works to me.

View solution in original post

0 Karma

xindeNokia
Path Finder

Found the solution for my question -

./splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging/values.yaml: path: /var/log/containers/.log
Changed to:
path: /var/log/pods/
.log works to me.

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...