Why do I get this error in splunkd.log? "SearchScheduler - Error in 'DispatchManager': The user 'splunk-system-user' does not have sufficient search privleges"

[torustad]

Seemingly after I installed the apps "Splunk App for Windows Infrastructure " and "Windows Add-on" the following error messages are flooding splunkd.log:

03-05-2015 15:48:51.448 +0100 ERROR DispatchManager - The user 'splunk-system-user' does not have sufficient search privleges.
03-05-2015 15:48:51.448 +0100 ERROR SearchScheduler - Error in 'DispatchManager': The user 'splunk-system-user' does not have sufficient search privleges.

I do not understand what they mean.

At the same time in the audit.log:

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD589d00151dd198770_at_1425566400_92588', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `dm_license_summary_10m_by_pool` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_49ee7cac9a05cfbf_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5758fe1b10509f00e_at_1425566400_92589', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `sourcetypes_summary_10m` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_463c95b47e289f0f_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5b13a71946e1b9d14_at_1425566400_92590', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `dm_license_summary_10m_by_source` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_6030b06d30f6e6f4_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5825f35f83c8311df_at_1425566400_92591', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `dm_license_summary_10m_by_host` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_26e747c470c62ba8_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5b453f9e7776e363e_at_1425566400_92592', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `sources_summary_10m` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_74744182914e20d1_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5cb7dc0fcb8381ee5_at_1425566400_92593', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `dm_license_summary_10m` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_28e51b5378d59f27_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD53af21b05a677c086_at_1425566400_92594', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `dm_license_summary_10m_by_forwarder` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Mar 05 15:40:00 2015', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_ef0750b59633eb8b_ACCELERATE_"][n/a]

03-05-2015 15:48:51.448 +0100 INFO  AuditLogger - Audit:[timestamp=03-05-2015 15:48:51.448, user=splunk-system-user, action=search, info=denied , search_id='scheduler__nobody_c3BsdW5rX2RlcGxveW1lbnRfbW9uaXRvcg__RMD5c2d3abf2a0486f8a_at_1425566400_92595', search=' summarize override=partial timespan= max_summary_size=52428800 max_summary_ratio=0.1 max_disabled_buckets=2 max_time=3600 [ search `indexers_summary_10m` ]', autojoin='1', buckets=0, ttl=60, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields='', apiStartTime='Fri Dec 05 00:00:00 2014', apiEndTime='Thu Jan 01 01:00:00 1970', savedsearch_name="_ACCELERATE_B4CAB4FC-62FD-4955-8951-F2777F04C839_splunk_deployment_monitor_nobody_3c6084d7f35794cf_ACCELERATE_"][n/a]

Thanks for any help,
Bård Tørustad

[torustad]

I found that the following lines had been added to .../Splunk/etc/system/local/authorize.conf:

[role_admin]
importRoles =
srchIndexesDefault = akseptanse;main;msad;perfmon;summary;windows;wineventlog;winevents
srchMaxTime = 8640000

After having removed these lines the messages above do not occur in splunkd.log anymore.

I do not know how those lines came to be added to authorize.conf (I did not do it myself explicitly :-)); maybe when I removed some app, or when I edited the role_admin - role to make the "Splunk app for Microsoft Windows infrastructure" work.

Should this happen at all?

[malmoore]

Does the splunk-system-user user have the winfra-admin role?

You should not need to edit roles manually - simply assign the winfra-admin role to the user that you log in as to run the app, and you should be done with that part.

[torustad]

Version information:
Splunk Version
6.2.1
Splunk Build
245427