You are using Windows OS for your DS and deploying to non-Windows servers.  This cannot be made to work.  Start over and use a Linxus OS for your DS. ... View more

This just means that the script being called did not run to completion. I recently had a confusing problem that caused this exact error. I had a working design where I had a sendalert named "my_send_alert" which called a python script named "my_send_alert.py" which then called a shell script named "my_send_alert_alt.sh".  It all worked great.  So I cloned it to create a different one and it didn't work, giving this error.  The problem ended up being that I named the shell script the same name as the python script and splunk was SKIPPING calling the python script and was calling the shell script directly!  I simply changed the name of the shell script and all was well.  So in summary: All 3 named the same does not work. This DOES NOT work: my_send_alert(alert_actions.conf) -> my_send_alert.py -> my_send_alert.sh This DOES work: my_send_alert(alert_actions.conf) -> my_send_alert.py -> my_send_alert_alt.sh This should also work: my_send_alert(alert_actions.conf) -> my_send_alert.sh ... View more

A better way to do this is to use the "Run a Script" alert action (after you create a script to do the copy).  Yes, this alert action is deprecated but I use it often and there is no way that Splunk will be removing it from the product. ... View more

According to the developer, it can be done with HEC: https://infosecwriteups.com/knowbe4-to-splunk-33c5bdd53e29 ... View more

You are reading his request backwards.  That git project is for SENDING TO OpenCTI.  He (and I) need to RECEIVE FROM OpenCTI.  I cannot find anything that does this. ... View more

WARNING! This answer is wrong.  The date of this file will be the date of the file when it was packaged in the installer (tgz/rpm). ... View more

WARNING!   The new website is broken and cannot display newlines inside of double-quotes, even as a "code snippet" which is also moronic and inexcusable.  So while my correct answer looked fine and worked in the old answers site, the new one cannot be made to display a correct answer so I am going to DESSCRIBE IT.  The space that you see in the double-quotes is actually supposed to be a newline as in <"></n><">.  If you do that, it works.  The stupid site is also forcing me to add an app, even this this Q/A has nothing to do with any app. ... View more

WARNING!   The new website is broken and cannot display newlines inside of double-quotes, even as a "code snippet" which is also moronic and inexcusable.  So while my correct answer looked fine and worked in the old answers site, the new one cannot be made to display a correct answer so I am going to DESSCRIBE IT.  The space that you see in the double-quotes is actually supposed to be a newline as in <"></n><">.  If you do that, it works. ... View more

Splunk has the mechanism in place to make it work: $trellis.name$ and $trellis.value$, but instead of these being available for ALL searches, they are only available for the trellis search/chart.  I have tried many things but all have failed.  If annotations are supported in Studio, you could try there.  That is my next step, but I don't know Studio so don't know if I will try or how long it would take.  The next best thing to do is make the "annotation_category" set to the value of the split-by field for the Trellis so that a hover makes it clear which ones are for which trellis panel. ... View more

This has finally been addressed in a useable way that seems to not have any downside/impact in 9.1 (search for "Preserve search history across search heads"): https://docs.splunk.com/Documentation/Splunk/9.1.1/ReleaseNotes/MeetSplunk Scarily enough, it appears to be enabled by default. ... View more

This has finally been addressed in a useable way that seems to not have any downside/impact in 9.1 (search for "Preserve search history across search heads"): https://docs.splunk.com/Documentation/Splunk/9.1.1/ReleaseNotes/MeetSplunk Scarily enough, it appears to be enabled by default. ... View more

Probably because there are also events (at least one) in that bucket that are younger. ... View more

There is so much "depends" here that we could open a nursing home.  Are you using SmartStore?  Are you using indexer clustering?  What are your SF/RF settings?  Are you using Volume settings for your indexers?  Are you Splunk Cloud?  What is the "btool" output for your indexes.conf from one of your indexers? ... View more

Yes, you could, but it really doesn't make sense.  You can just directly peer the new Search Head to every/any Indexer.  You can discriminate what lives/came-from where by examining the "splunk_server" value.  So whatever you do, you can add a splityby (... BY splunk_server) to keep the results separated. ... View more

See my answer.  The accepted answer is useless. ... View more

... | eval 2nd = mvindex(mvfield, 1, 1) ... View more

A while back I noticed the "endpoint" setting in the "serverclass.conf.spec" which *probably* could be used to create a functional Windows-based DS by directing the clients to retrieve the files from a non-Windows repository such that file modes are preserved: https://docs.splunk.com/Documentation/Splunk/latest/Admin/Serverclassconf ... View more

To exercise the rest endpoint in its most basic use. ... View more

The search above is slightly wrong.  Try this: | rest /services/search/jobs splunk_server=local | stats count avg(performance.command.addinfo.duration_secs) AS avg max(performance.command.addinfo.duration_secs) AS max BY search | sort 0 - max - avg ... View more

https://www.youtube.com/watch?v=1yEhbKXRFMg ... View more

What is your approach now? ... View more

What works?  You replied to yourself saying that it doesn't work???? ... View more

This is a stupid question on many levels.  You should NEVER use the "date_*" fields for anything because they are pre-TZ-normalized values and do not take into account your user's personal TZ value and so almost always will be incorrect.  You should instead calculate your own "date_*" values, if you even need them (which in this case, you do not).  The question is phrased EXTREMELY poorly and is very confusing to understand.  I spent 5 times as much time reading and re-reading the question as I did coming up with the answer.  I suspect that part of the answer is that you should know (and explain in your answer) that by default, only the last 30 days are stored in the internal index data so this question is even dumber because you will never have a complete month in your data.  In any case, if I understand it correctly, here is one solution: |tstats count AS Count WHERE index="_internal" AND date_month="*" earliest=0 latest=now BY host _time span=1d | rename _time AS Month, host AS Host | eval _mon=strftime(Month, "%Y%B") | streamstats current=f last(_mon) AS prevmon BY Host | eval neweventhere = if(_mon!=prevmon, mvappend("FIXME","KEEPME"), "NO") | fields - prevmon | mvexpand neweventhere | rename neweventhere AS _neweventhere | eval Month = if(_neweventhere="FIXME", (Month - 1), Month) | eval _mon=strftime(Month, "%Y%B") | eventstats sum(Count) AS _TotalCount BY _mon Host | eval Count = if(_neweventhere="FIXME", _TotalCount, Count) | fieldformat Month = if(_neweventhere="FIXME", strftime(Month, "%B"), strftime(Month, "%B %d")) | table Month Host Count _* ... View more