We have a number of jobs running as "admin" that run and create large temporary files on disk and when the disk quota kicks in we aren't certain if it is due to this job, or another job running as admin. Since this is a server shared between multiple development teams, I don't want one teams search to impact other teams ability to debug their code.
In the past, a fellow admin has changed the owner to "nobody" to get around the quota problem without resorting to increasing a quota - apparently "nobody" does not have any quota restrictions?
My thought is to change the owner of the job to the name of the developer or team that created it and work with them to either resolve the quota issue, or increase their quota to allow these jobs to run.
Here are my questions:
I'm ok with using "nobody" to work around the quota restrictions for a short time if that works. I can't find a document/wiki/answer that addresses what restrictions the "nobody" owner has - can anyone help?
How have others addressed this? I'm tempted to create a "team account" that is just for running that teams jobs while keeping the ability to control run-away jobs in check. Are there other options I've overlooked?
Is the "splunk-system-user" an appropriate owner for these jobs? My gut says no since it's usually for internal/system jobs and could be as bad as "admin".
This is pretty old but, in newer versions of Splunk, you can delegate who a report runs as owner or user. If you select the user option, the job will run as the user thus respecting the quota's for that user and role. If your teams are assigned to different roles, this will enable you to ensure that team a's jobs do not impact team b's.
If it's scheduled it must run as owner. Could modify the metadata files to shift ownership or remove it entirely (nobody).
The run as owner or user referenced above I believe is for dashboards and the running of searches. You can build the dashboard to run the search contained within it, which runs as the user accessing the dashboard. Or you can set up the dashboard to reference the saved search which can run as the owner for that search.