That's a good point, wouldn't hurt to use the support! That makes sense that the sendRcvTimeout might not be the real root of the problem. Those two symptoms sounds pretty accurate, I'm going to dig some more into this. Thank you very much for the all the insight and help! ... View more

@hattrickNZ - did this answer your question? If so, you can "accept" this answer or post clarifying comments. ... View more

@sagineshmk, did this answer your question? If so, you can "accept" this or provide clarifying information to continue the discussion. ... View more

Interesting. I used to have issues with SHP where the mount would get lost and it was all sorts of bad news for Splunk. Here's a zany idea: I've used a scripted input of Splunk to restart Splunk. I don't know how much of the scripted input is kept in memory but you could maybe try an experiment of pushing out a scripted input job that runs after the work should be done and just does a splunk restart for the forwarder. For the experiment, try deploying the script to a different filesystem (using the deploymentclient or serverclass settings) so that even when the mount is gone, the splunk forwarder running in memory can still load the restart script. Some other options: an enterprise job scheduling tool? remote ssh has some nifty features for running the same commands remotely across many hosts make sure splunk is a boot service and ask them to reboot the machines after their work ... View more

Good, simple solution. Thanks. ... View more

That worked Its always the simple stuff. Thanks! ... View more

great, i'll give it a shot. thanks for the help! ... View more

If it makes it more clear, the problem is that "transaction a,b" is an "AND" match, and I need an "OR" match. ... View more

take a look in the $SPLUNK_HOME/etc/system/default/transforms.conf file... there are transforms you can manipulate to further break down the uri_path into root, file etc... I use that to good effect on these type of logs. you don't have to do it yourself... ... View more

Making this change in distsearch.conf is the "Big Giant Hammer" solution to this...I suggest trying my2ndhead's steps below since that tries to address just the hiccup that caused this problem. This way you get rid of the bad-stuff that happened to be created instead of hiding the problem behind a setting change. If you have big bundles, turning off the delta functionality will cause unnecessary overhead if the root cause was a minor hiccup in the bundle delta replication process. ... View more

Correct! I was obliterating _time. Built-in time functions, report back 8 days in a week, probably b/c of timezones. latest(_time) didn't work over multiple weeks, the below query while not pretty does the job. ... | eval WorkWeek=strftime(_time, "%U")| eval my_day=strftime(_time, "%d") | eval Day_ID = my_day+";"+WorkWeek+";"+UserID | stats dc(Day_ID) AS DaysConnected by WorkWeek, UserID | stats count(eval(DaysConnected<=3)) AS OccasionalUsers count(eval(DaysConnected>3)) AS FrequentUsers by WorkWeek | eval UserLoyalty = FrequentUsers*100/(FrequentUsers+OccasionalUsers) ... View more

So you don't need the indexer_lag field, per se, but your overall search will be similar. If you're looking at specific sourcetype over all indexes, then your search may start like this: index=* sourcetype=tfitzgerald15s_type | And the indexer_lag is just a new field I am calculating based on what I want to base my threshold on for the example. So in your case it might be a calculation you have to do for CPU usage, or HTTP response times, or transaction duration. Also, if this answers your question don't forget to accept/up-vote the answer 🙂 ... View more

Are you running this on a unix server? I'll assume so, mainly because that is what I am familiar with - if it is Windows then I am not so sure. The script doesn't have to be anywhere special, you just need access to the splunk and python commands via your path. You should be the user that the splunk application is installed as (so log in as that user, sudo to it, etc), and the splunk command should be in your path (i.e. the one installed in [SPLUNK_HOME]/bin/). Finally, python should be in your path, too - chances are they python you will be using is in [SPLUNK_HOME]/bin/ as well. You can check from the command line where splunk and python are by doing: which splunk which python Both of those should return you the path where those are. Now you can run the script from any location you want - all the script does is iteratively invoke the splunk CLI with your search for the interval/time-range you modified the script to be in. That would be something like the following command: python mySummaryScript.py One suggestion if you haven't done it already - create another copy of your script that only summarizes 3 or 4 intervals over a smaller time range. Then have the script just dump this into index=summary. Run that one first, wait a few minutes for the summary data to show up, and then verify in index=summary that things look good before running your full script that populates your real summary index. This way you don't inadvertently put in bad data to your real summary if you have a typo. ... View more

this is probably the right answer and thanks for the help. I guess this is just a little out of my reach. I don't know how to create the .js widget from the code sample. Is the .js just a text readable file with the .js extension? Also I need the numbers to be color "white" since the clock is on a black background and I need them to be a larger font. Please forgive me I'm learning here. Thanks again for your help. ... View more

How to achieve this if I want to navigate to a folder path Consider the result of splunk query Name| path Path1 | \abc\p1 Path2 | \abc\p2 I want to click and navigate to the paths . For weburls i can achieve using $click.value2|n$ But in the case of folders and shared paths how to achieve this??? ... View more

eval Comment=if((Fund_Credit_Amt=Ack_Credit_Amt) AND (Fund_Debit_Amt=Ack_Debit_Amt) ,"MATCH","MISMATCH") ... View more

Quick note that the dec() in the example above should be dc() ... View more

Just a bit of follow-up on this. One reason against PDF Server everywhere is the app is pretty big with a 32-bit and 64-bit version of Firefox internal to the app. So depending on your build/deploy process and how things are with Deployment Server you might not want to be pushing PDF Server out all over the place. On the other hand, it shouldn't necessarily change often so if you're using Deployment Server it won't often try and push out copies of PDF Server. ... View more