So I'd like to have one custom rest handler that allows for URLs like
script = myapp_files.py
scripttype = persist
I would presume that if the docs don't say a regex is allowed then a regex is not allowed.
There are examples, however, of pattern use in the OOTB restmap.conf files. See splunk_instrumentation/default/restmap.conf for an example. In your case adding a slash may help.
If it works then submit feedback on that documentation page to ask them to say patterns are allowed.