loadjob command not working

[kaeleyt]

I have a need to share high level metrics (via tstats) from a couple of indexes that a few of my teammates do not have access to. I have a scheduled report, let's call it ScheduledReportA, that is running that tstats command once a day in the morning.

I was planning to use the loadjob command to load the results of that report into a dashboard that my teammates can then filter on and search to get the information they need but I've noticed that the loadjob command only works some of the time for me, and otherwise will return 0 results. I know it is not my search syntax as I have used the same search and sometimes gotten results, sometimes not. Syntax for reference:
| loadjob savedsearch="kaeleyt:my_app_name:ScheduledReportA"

Some additional information to help rule things out:

• The loadjob command search is being run in the same app that ScheduledReportA lives in
• The report always has thousands of results, and yes I've checked this
• ScheduledReportA is shared with the app and its users
• dispatch.ttl is set to 2p (which I have always understood to be twice the schedule, which in this case is 24h, so 48h ttl)

I don't suspect it to be a permissions issue, or a job expiration issue based on the above but I'm wondering if I'm missing something or if anyone has run into similar issues.

[livehybrid]

Hi @kaeleyt 

I wonder if the following will help work out what is going on? Can you run this to see if this shows the resultCount=0 or any other issues?

You might need to tweak:

| rest splunk_server=local /servicesNS/nobody/my_app_name/saved/searches/ScheduledReportA/history
| table updated, published, eventCount, is* id
| rex field=id "(?<uri>\/services.*)$"
| map  maxsearches=10 search="|rest $uri$ "
| table id dispatchState eventCount resultCount ttl is*

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[kaeleyt]

The ttl of my latest job for this report is set to expire approx 2 days (~170000+ seconds) from now. I've been using the _audit index to check the resultCount for these jobs and it has never been 0. I've also been checking the ttl via the Activity > Jobs view (2nd image in screenshot, had to take a screenshot of my two screenshots to get past the 1 attachment limitation 🙂 ).

Is it possible that having multiple saved jobs from this search alive at the same time causes an issue? I have two alive currently, one from yesterday morning and one from this morning (similar to what you're showing in the screenshot). If this is a possibility, any recommendations on how I can only have 1 report job alive/kept at any point in time?

[isoutamo]

Are you running this on SH or SHC?
Are your result set events or transformed results?
https://docs.splunk.com/Documentation/Splunk/9.4.2/SearchReference/Loadjob

[kaeleyt]

Running on SHC and it's transformed results.