Since I migrated splunk to version 9.2.4, I've been getting a lot of error messages from all Splunk servers : WARN UserManagerPro [16791 SchedulerThread] - Unable to get roles for user=nobody because: Failed to get LDAP user=“nobody” from any configured servers ERROR UserManagerPro [16791 SchedulerThread] - user=“nobody” had no roles I think these are all scheduled searches that are executed without an owner and therefore executed as user nobody. These messages didn't appear with version 9.1 What's the best way to turn off these messages? The annoying thing is that some searches come from Splunk apps (console monitoring, splunk archiver, etc.) ... View more

My deployment consists of 2 servers to collect syslog sources. On each server is installed a rsyslog daemon that receives messages in UDP and spool them into log files. These files are monitored by a universal forwarder which sends the messages to indexers. This deployment is a good practice for indexing syslog data. A LB F5 is installed on the front end and routes the flows to both servers. I wanted to set up a mechanism that would allow me to manually add or remove a universal forwarder from the member pool when it is under maintenance or restarted for example. For a search head cluster it is possible by configuring a custom endpoint like the suggested solution https://community.splunk.com/t5/Monitoring-Splunk/F5-Load-balancer-Pool-member-health-monitor/m-p/459497 But it is not possible with a universal forwarer by design (the python library is not embedded) for security reasons So my question is how to manually disable a universal forwarder so that the server does not receive any more data from the LB? ... View more

Hi everybody I have a challenge I must to deal with. I would like to create a script to deploy application from en to end. This script must execute many steps : 1) copy app in the deployment-app directory on the deployer 2) reload the configuration server (reload deploy-server) 3) wait until the app deployment is finished 4) return the code status deployment in output the script can be called by a human or a external tool. I know that query on metadata index can allow to follow the app installation on the clients : index=_internal component=deployedapplication OR component=deploymentclient | sort host _time | table host _time component message But this dont tell me if the app is correctly deployed or not I wonder if anyone has written such a script ? Anyone can guide me or give me advice ? Regards ... View more

Hi everybody Can you help me and suggest me a solution about my context. I have many servers Jboss, and each server host many instance/JVM (1 to 4). Example a server jboss_server1 hosts 2 instances jb_instance1 and jb_instance2. An instance is identified by a search time field JBOSS_INSTANCE. I would like scheduled a search to monitor the daily volume log for each instance and trigger an alert when the volume exceed a daily quota (300 MB for example) To do this I write a search : index="jboss" earliest=-0d@d [search index=_internal source=*license_usage.log type=Usage earliest=-0d@d | stats sum(b) as bytes by h | rename h as host | where (bytes/1024/1024)>300 | fields - bytes] NOT ([|inputlookup high_volume_jboss | fields JBOSS_INSTANCE]) | fields + host,CTX,JBOSS_INSTANCE | eval raw_len=len(_raw) | stats sum(raw_len) As TotalSize by host,CTX,JBOSS_INSTANCE | eval TotalSizeMB=round(TotalSize/1024/1024,2), quotaMB=250,newLogLevel="ERROR"| where TotalSizeMB > 250 I make search with a join on the license_usage.log to match the host which log more than 300MB (don't forget i want sizing by instance not by host) I make a second filter to exclude instance that is matched in a input lookup containing pair host/instance matched by a precedent search in the day I calculate the size of each raw matched But I think this is not optimized. What do you think ? ... View more