The splunk(-company)-wrapped syslog-ng service, "Splunk Connect for Syslog" (AKA SC4S) comes standard with a systemd unit file that reaches out on every startup to github to obtain the latest container image. This had worked flawlessly since we first setup syslog inputs for the client. However years later, somebody made a WAF change that blocked connectivity to github, which included our download URL round in the unit file (specifically, ghcr.io/splunk/splunk-connect-for-syslog/container3:latest) and did not properly warn or socialize this fact before doing so. This caused the sc4s service to be unable to restart because the systemd unit file downloads a fresh image every time before it starts, which it could no longer do. WARNING, if you setup SC4S the normal way, then you did so as user "root" so you will need to do all of this as user "root" also. The most immediate solution is to see if there is still an older image around to run by using this command: docker image ls You should see something like this: REPOSITORY TAG IMAGE ID CREATED SIZE ghcr.io/splunk/splunk-connect-for-syslog/container2:2 latest SomeImageID2 SomeDate SomeSizeGB If there is, you can modify the unit file by copying the "IMAGE ID" value (in this case "SomeImageID2") and changing this line: Environment="SC4S_IMAGE=https://ghcr.io/splunk/splunk-connect-for-syslog/container2:2:latest" To this: Environment="SC4S_IMAGE=SomeImageID2" And also commenting out this line, like this: #ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE Then you need to reload systemd like this: systemctl daemon-reload This should allow you to start your service immediately as normal: service sc4s start Now you have the problem of how do you get the latest image manually (now that the automatic download cannot work) which according to this link: https://splunk.github.io/splunk-connect-for-syslog/main/upgrade/ is now this: ghcr.io/splunk/splunk-connect-for-syslog/container3:latest The following link gave us all of what we need but we had to do it a few times with various options mined from the comments to get it eactly right: https://stackoverflow.com/questions/37905763/how-do-i-download-docker-images-without-using-the-pull-command You will first have to install docker someplace that CAN get to the image URL. If you can run a broswer there, just post the value in your browser and it should redirect to an actual page. If you only have the CLI there, just use curl to test like this: curl ghcr.io/splunk/splunk-connect-for-syslog/container3:latest In our case, we just installed docker on a Windows laptop and then opened powershell to run these 2 commands: docker pull ghcr.io/splunk/splunk-connect-for-syslog/container3:latest docker image ls You should see something like this: REPOSITORY TAG IMAGE ID CREATED SIZE ghcr.io/splunk/splunk-connect-for-syslog/container3 latest SomeImageID3 SomeDate SomeSizeGB Next you need to export the image to a file like this: docker save SomeImageID3 --output DockerImageSC4S.tar Then transfer this to "/tmp" on your SC4S server host however you please and load it like this: docker load -i /tmp/DockerImageSC4S.tar Then, of course, you need to re-modify the unit file using the new "SomeImageID3" value instead of "SomeImageID2".
... View more