Session lock files

[mslvrstn]

All of my indexers have thousands of session lock files similar to

-rw-------  1 root   root         0 Jan 27 16:38 session-fa9570368b47a81458a696720fb657239dc0a60c.lock

ls /opt/splunk/var/run/splunk/session*.lock | wc -l
20379

which go back for months.

I don't know that there is anything necessarily wrong with this, but it doesn't feel right. Is this common? If it isn't, what kind of misbehavior should we be on the lookout for?

[johnhsu]

Remove tons of session files
ll | grep session | awk '{print "rm "$8" "}' | csh

Note:
"$8" -- column 8 of ll comand

Thanks
Sincerely
John Hsu

[gareth]

Well even with that traffic, the lock files shouldn't be months old; something odd there.

[mslvrstn]

Firstly, thanks for the edit, dwaddle. I see what you did with the indent and I will do that from now on.

Gareth, it's 4.1.6 on Linux. Your question about web_service log was right on target. At first I thought not, since these are indexers and don't get much http traffic, although splunkweb is enabled for convenience. Then I looked at the logs and saw tons of web vulnerability assessment traffic. Understanding that those session files correspond to splunkweb traffic was the key. Thanks!

[gareth]

No, that shouldn't happen. Extraneous session and lock files should be purged every five minutes or so.

Which version of Splunk are you running (and on what platform)? Are there any related error messages in the web_service.log?