Splunk Dev

Python Error in restarting Splunk

teddyidc1101
Communicator

upon restarting splunk forwarder, we are encountering the error below..

python: can't open file '/opt/splunkforwarder/lib/python2.7/site-packages/splunk/clilib/cli.py': [Errno 2] No such file or directory

what could be the reason and fix for this?

Tags (2)
0 Karma

xpac
SplunkTrust
SplunkTrust

Hey, try running this:

/opt/splunkforwarder/bin/splunk validate files

It will check your install for missing files, and give you a hint what is wrong.

teddyidc1101
Communicator

By the way, here are the items we looked into to check...did we miss anything?
- Checked the logs and confirmed that the events stopped coming in at 5/21 around 12:30MN
- Check on crash logs…none found
- Checked forwarder management, the server was not a client
- Executed Splunk start forwarder instead of Splunk restart - failed
- Obtained the following logs in /opt/splunkforwarder/var/log/splunk of the server : python.log – not found for splunkforwarder, splunkd-utility.log, splunkd.log
- When we checked the actual file the restart is looking for, file was not found, as indicated in the error does not exist -
python: can't open file '/opt/splunkforwarder/lib/python2.7/site-packages/splunk/clilib/cli.py': [Errno 2] No such file or directory
**Does this file really exist?

What is the best way to revive the UF.**

0 Karma

MuS
Legend

Did you see my comment I posted just one minute before you posted your comment?

0 Karma

teddyidc1101
Communicator

yeah i did...did we miss any steps with the one listed above?
the client installed the UF in their servers. we dont have access on them. but upon checking via splunk search, UF version is 7.0.

checking the UF , we dont have the Python in there but what the team did was linked python and python 2 to the /opt/splunkforwarder/bin. Is this ok?

upon checking in server via command "whereis python" , it listed /opt/splunkforwarder/bin/python.
is this ok?

0 Karma

MuS
Legend

you did what???
Please revert those changes. The universal forwarder does not need Python by default. It only needs Python if you want to run a python script using the universal forwarder.

Still this is really weird .....

teddyidc1101
Communicator

ok thanks...the linking happend 5/10 and UF stopped in 5/21, so i am not sure if they are connected at all...if we remove the link, does it impact the start/restart of the splunk instance? will it restart successfully? is reinstalling the forwarder an option here?

0 Karma

xpac
SplunkTrust
SplunkTrust

Yeah, that's sounds rather strange... If I were you, I'd wipe the complete /opt/splunkforwarder directory and reinstall the forwarder from scratch. Be aware, however, that this might lead to duplicate events for some time.

0 Karma

teddyidc1101
Communicator

ok thanks @xpac...i'm already considering it as i'm running out of resources to look into 🙂 thanks so much for sharing you expertise!

0 Karma

teddyidc1101
Communicator

I triend running the command and its saying that
Result: Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-x86_64-manifest'
All installed files intact.

0 Karma

MuS
Legend

Wait a second, you get this error /opt/splunkforwarder/lib/python2.7/site-packages/splunk/clilib/cli.py': [Errno 2] No such file or directory ... but this is a universal forwarder ?!
Splunk universal forwarder does not ship Python, the only Splunk software that ships Python is the full enterprise install.

How did you install this version of Splunk; What package did you use and what command to install it?

cheer, MuS

0 Karma

MuS
Legend

did you use the /opt/splunkforwarder/bin/splunk start command?

0 Karma

teddyidc1101
Communicator

yes we did but still getting the error.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...