Hi @venkatanagendra , You do not need to do regex, as your data is already in  structured json format. Ensure your sourcetype is using kv_mode=json  or kv_mode=auto and you can address fields  ErrorCodes{}.errorCode ErrorCodes{}.errorMessage Best regards, Andreas   ... View more

Hi @pdafale_avantor , i hope i understood your question correctly: you want to recreate identities.csv every week from ldap.  You already have your ldap query, so it would be as simple as using the outputlookup command. This will create the lookup in the current app and overwrite the existing identities.csv | ldapsearch domain=SPL search="(objectClass=user)" | stats something | outputlookup createinapp=t indentities.csv best regards, Andreas ... View more

Hi Ra1n, the | rest command gives you the state of an endpoint - in this case savedsearches. Dit does not - what you might expect - gives you a history. That's why timebases searches are not working by default.  if you want an audit on your correlation searches, i would suggest implement a cicd workflow and have the app and/or savedsearches.conf file version controled... particularly if dealing in security environments.  best regards, Andreas       ... View more

Hi, i would go with this event in splunkd.log: "IndexProcessor [5762669 MainThread] - request state change from=RUN to=SHUTDOWN_SIGNALED" this is triggering Splunk to shutdown..  last events at shutdown looks like this: 10-28-2022 16:35:56.890 +0100 INFO Shutdown [5763136 Shutdown] - shutting down level="ShutdownLevel_Duo2FAHttpClient" 10-28-2022 16:35:56.890 +0100 INFO Shutdown [5763136 Shutdown] - shutting down level="ShutdownLevel_S3ConnectionPoolManager" 10-28-2022 16:35:56.891 +0100 INFO Shutdown [5763136 Shutdown] - shutting down level="ShutdownLevel_WorkloadManager" 10-28-2022 16:35:56.894 +0100 INFO loader [5762669 MainThread] - All pipelines finished meaning shutdown could be measured with this: index=_internal sourcetype="splunkd" source=*splunkd.log (request state change from=RUN to=SHUTDOWN_SIGNALED) OR (Shutdown shutting down level=*) | transaction startswith=SHUTDOWN_SIGNALED | table _time duration   best regards, Andreas   ... View more

hi @LogUx , you are searching for the workload management feature @chaker already provided the links. regards, Andreas ... View more

hi, This sounds like a working solution. Me personally would first try to create an alert instead of report and set the trigger to "send email" there you can select "attach CSV".. should do the same with one object to maintain. best regards, Andreas ... View more

Hi, doing an eval from a second search is working like this - same like yours but "| return $fieldname" added | makeresults | eval indexcount=[| tstats count as indexcount where index=_internal earliest=-24h@h | return $indexcount]   but, i guess this would not work in your token you are passing down from main search.. regards, Andreas ... View more

Hi,  when searching via API you are providing parameters: cparams = { "host" : lconfig["host"], "port" : lconfig["port"], "username" : lconfig["user"], "password" : lconfig["password"], "app" : lconfig["app"] } most propably you are not using same user or correct app context.  check index=_audit and/or index=_internal sourcetype=splunkd_access + username.. best regards, Andreas ... View more

Hi Venkata, Please use the documented ACS Endpoint:  https://admin.splunk.com/STACK_NAME/adminconfig/v2/indexes https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Config/ManageIndexes This works for sure! 😉 Cheers, Andreas ... View more

Hi Venkata, IMHO you have to use ACS and not Splunkd for that https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Config/ManageIndexes what endpoint are you using. How does the reponse look like? regards, Andreas ... View more

Hi, if you are sharing multiple roles on one instance you can also disable DS by setting serverclass.conf   [default] disabled = 1   https://github.com/schose/splunkspecs/blob/09c9e391b692e9c23e470f2e6dac475bbfd80419/README/serverclass.conf.spec#L48 we currently enable DS for a short time frame to mitigate the issue. regards, Andreas  ... View more

Hi, so if it's really only splunk btool you can run splunk btool using a scripted input. This will run the command as the splunk user. you can capture the output.. no problem. done this before. regards.  Andreas ... View more

hi, you can try to verify using license usage: index=_internal source="/opt/splunk/var/log/splunk/license_usage.log" sourcetype=splunkd earliest=-7d@d latest=-6d@d | stats sum(b) as lic_7d | append [ search index=_internal source="/opt/splunk/var/log/splunk/license_usage.log" sourcetype=splunkd earliest=-1d@d latest=-0d@d | stats sum(b) as lic_yesterday ] | stats values(*) as * | eval pct=round((lic_yesterday/lic_7d)*100)   license usage is also a good indicator for ingest - could be just misleading if you are doing a lot of metric conversion..  ... View more

Hi, not sure if i get you correctly - you want to talk to splunkd using a scripted input without having credentials?generally "splunk btool whatever" would be a bad example, as this does not need splunkd to be running.  <input> <server_host>art-macbook.local</server_host> <server_uri>https://127.0.0.1:8089</server_uri> <session_key>YmGDj6BtVw^dAb1UzmBjg8MwOegMXHeNtF17THnAoR0Ot2HXQ7BXZ9mPI^hNkdFN^yyTYvrPxjf0WThDmW9sahRomNrj^t^KYG8V30hE9gaPh2gVV7H0LnY</session_key> <checkpoint_dir>/Users/andreas/splunk/var/lib/splunk/modinputs/buba</checkpoint_dir> <configuration> <stanza name="buba://single" app="buba-backend"> <param name="configfile">buba.conf</param> <param name="disabled">0</param> <param name="env">singlerestore</param> <param name="host">$decideOnStartup</param> <param name="index">default</param> <param name="interval">80000</param> </stanza> </configuration> </input>   you should avoid scripted input, as they are deprecated. when you install a modular input instead your script is started and hand over a xml payload and you will find a session_key there. You can use this sessionkey to talk to splunkd.   regards, Andreas ... View more

the cheaper way is to use _metrics index | mstats sum("spl.mlog.thruput.thruput.instantaneous_kbps") as total_7d_ago WHERE "index"="_metrics" earliest=-7d@d latest=-6d@d host=INDEXER BY host | append [ | mstats sum("spl.mlog.thruput.thruput.instantaneous_kbps") as total_yesterday WHERE "index"="_metrics" earliest=-1d@d latest=-0d@d host=INDEXER BY host ] | stats values(*) as * by host | eval pct=round((total_yesterday/total_7d_ago)*100,1) you have to place you indexers with or in where part. regards, Andreas ... View more

Hi, Can you be more precise what at your splunk environment looks like? If you mean you are running multiple single search head instances there is no communication between the kvstores and migrating vom mmap to wiredtiger one-by-one shouldn't be any issue.  if you are kvstore heavily or big dataset try performance in a test environment. We have seen increases in search time up to 3x.  best regards, Andreas  ... View more

Hi, i guess i would use splunkd metrics for that : index=_internal sourcetype=splunkd source="/opt/splunk/var/log/splunk/metrics.log" "group=thruput" "name=index_thruput" instantaneous_kbps earliest=-7d@d latest=-6d@d | stats sum(instantaneous_kbps) as ingest_pipe_instantaneous_kbps by ingest_pipe host | stats sum(ingest_pipe_instantaneous_kbps) as total_7d_ago by host | append [ search index=_internal sourcetype=splunkd source="/opt/splunk/var/log/splunk/metrics.log" "group=thruput" "name=index_thruput" instantaneous_kbps earliest=-1d@d latest=-0d@d | stats sum(instantaneous_kbps) as ingest_pipe_instantaneous_kbps by ingest_pipe host | stats sum(ingest_pipe_instantaneous_kbps) as total_yesterday by host ] | stats values(*) as * by host | eval pct=round((total_yesterday/total_7d_ago)*100,1)   in this example we calculate the sum of instantaneous_kbs per indexer per ingest_pipeline 7 days ago and compare it to yesterday. you might want to adjust times..    best regards, Andreas ... View more

btw. here is an example for docker-compose using direct filesystem mapping version: '3' services: single: image: splunk/splunk:8.1.5 ports: - "8111:8000" volumes: - single-etc:/opt/splunk/etc - single-var:/opt/splunk/var - /my/path/to/indexapp/indexapp:/opt/splunk/etc/apps/indexapp hostname: idx1 environment: - SPLUNK_HOME=/opt/splunk/ # - DEFAULTS_URL=http://splunk-defaults/default.yml - SPLUNK_START_ARGS="--accept-license" - SPLUNK_PASSWORD=EnterYourCreditCardNumber - SPLUNK_ROLE=splunk_standalone - SPLUNK_DEBUG="true" volumes: single-etc: single-var: networks: default: external: name: splunk   regards,   Andreas ... View more

Hi @isoutamo , that's exactly what i planned for my next article. But i can share the first numbers: After ingest 800MB raw routeros logs into event indexes the rawdata is:  gzip:  74.2MB  lz4: 136.0MB zstd: 56.4 MB I'll need some more time for a performance review, but will post updates.  regards, Andreas ... View more