Also, are you trying to install Splunk UF version 6.3.1? I heartily recommend using a newer version. Is 6.3.1 even supported any more?
So, after your ... "thing" that you are doing you are getting a "/opt/splunkforwarder/bin/splunk" not found. So, have you looked at your filesystem and seen where it really is? Is it where it's supposed to be, or missing?
If it's not there, then obviously something in your automation is not working right - debug your automation, I don't think there's a Splunk problem.
If it IS there, then check permissions, ownership, executability - all those things that you'd check if you were sitting at the console trying to run it and it said "not found". This may or may not be a Splunk problem, but likely is a problem with your chosen Linux distribution (being, if I looked it up right, a "security oriented, lightweight distribution" immediately indicates to me that "your stuff may or may not work, because we may have locked it down too tight or we might have not included critical libraries your app needs".
I do agree nothing I see in the config seems to be obviously off assuming "tar xvz -C /opt" actually extracts to /opt (I always just cd /opt, then tar xzv in there.)
Also, it may or may not matter in this case, but the uid/gid of the directories created after extracting are 506, so you may need to chown them to root?