I have a python script which returns all kinds of images via REST interface by going for some external sites to fetch them first. Apparently, the results of all such requests to the external sites are cached in $SPLUNK_HOME/bin/.cache/ . Unfortunately, our requests are constantly changing, so, on one hand, we don't really need that cache much, and, on the other hand, that folder is growing because each request is a new one. At some point, Splunk stops all searches because the root partition has less than 5G free space. That's how we discovered bin/cache - by running around the whole system with du and looking for the offending folder. Are there any configuration settings allowing us to cap the size of bin/.cache somehow? I couldn't find anything in Splunk Admin manual. ... View more

We have a well established Splunk app on an instance which is serving as a Search Head and an Indexer. However, there are some data there which needs to be forwarded to some other site, which hosts a different application. Some of the data comes from a modular input (receiving some TCP traffic), but there are others, like *hix TA, which we would also like to forward to that other app at a different site. Is there any trick to do that? Any special settings I need to have in inputs.conf and outputs.conf to work properly and not disturb the main operation, which has quite a few indexes and wants its data locally? ... View more

I have data which contain a field with a lot of values and has duplicates on almost every one - a barcode, scanned in more than one place. In addition to being a part of a long field (a kind of a combined tag) like <fixed-length prefix><%03d length><barcode of length-3> , that barcode does appear on its own in a couple of places in the same event, which hopefully means I can search on it quickly enough - though what format of the search string to use to optimize it, I'm not sure. The event might contain several such barcodes, and the "main" one is actually the one from that combined tag I mentioned earlier. I can't predict whether that barcode will appear as barcode1, barcode2 or barcode3 (the actual field names are different, but you get the idea), so even that is not so straightforward. However, there is a bigger problem: one of the search terms our application must have is a track ID, which is a trailing part of the barcode, extracted by some special rules (which I can easily put into eval command, if necessary) and never appearing directly in the raw event. For example, we might have a barcode 310200549315, which will appear in barcode2 (barcode1 and barcode3 will have different values and be of no interest to our application) and be a part of the combined_tag=PRD015310200549315. The track ID, as a suffix of the barcode, might have a value of 49315, which is always only a part of a token inside of the event. Even if I succeed in creating a calculated field track_id, searching on it will be excruciatingly slow. And searching on *49315 (continuing my previous example) will be no better. My question is: how would you attack this problem? I'm ready to create an index-time track_id field, but can I put an EVAL somewhere in props/transforms to achieve that? Alternatively, is there a way to optimize the search for such situations? ... View more

I'm trying to solve the following problem: in our client's environment, the clocks on different servers can vary greatly. We can easily have a server which is 3 hours behind on its system clock. And it's not a timezone issue - the servers are all supposed to have the same time. All servers have univeral forwarders sending events to some Splunk Enterprise instances. We want each event to have the time of its import by the universal forwarder as a timestamp. Will setting DATETIME_CONFIG=CURRENT on the forwarder and DATETIME_CONFIG=NONE on the indexer produce the desired result? ... View more

I'm submitting a search through splunklib (PythonSDK). On the output side, I need some fields which are all either aliased or calculated ones. So I figured I'd supply the rf search parameter along with the rest of the needed stuff (like "search", "exec_mode", "earliest_time", "latest_time" etc.). I made sure I connected via client.connect with the app keyword set to the application where those field extractions and aliases are defined. In addition, I tried to specify namespace search parameter, again supplying the name of the application. No luck - the results (extracted via ResultsReader ) never contained any of those fields I specified in rf . I tried both comma-separated and whitespace-separated, quoted and unquoted list (like field1, field2, ... , "field1" "field2" ... etc.) to no avail. Anything else that I'm missing? ... View more

I want to bind an action to a click event (probably 'click:cell', but can go with 'click:row') in a TableView. That action will get the data from the click event and display a popup (Bootstrap modal) populated according to those data. All this is done in an HTML panel. I know how to get the value of any column in the table from the event. However, I would like to have some data which are not shown on the screen, but are still "bound" to a table row. Even an access to a _raw field (as long as it's not displayed) would suffice - I'd extract what I need in JavaScript. To clarify - let's consider the following search: index=myIndex category=$selected_category$ | fields product, cost, amount, total, category | table _time, product, cost, amount, total $selected_category$ is a token set by a drop-down and having a value of "*" (All) or a legitimate category value (populated by a search piped into dedup category ). As you can see, I do not want to display category on the screen. However, I would like to have access to it from the click event. Right now, all the various data.row.<field> and event.rowContext.row.<field> members do not include that category field. Any reasonable way to achieve what I want? ... View more

We've used <selection> tag quite successfully for our timecharts. The regular <selection> <set token="slider.start">$start$</set> <set token="slider.end">$end$</set> </selection> addition to a timechart populates the slider.start and slider.end tokens quite fine each time a user makes a selection or pans it. They get the starting and ending epoch times, respectively. However, the same approach fails miserably for non-time charts. If, say, a span of some column is 965 to 970 of independent variable, and that column happens to become the first one in the selection, we'll get "965-970" inside the $slider.start$ token. How can we set the token properly - extracting that 965 from it? This panel is going to be exported to HTML at some point, but is there an easier way for now? We still have a lot of development ahead of us. ... View more

We have a fast growing index which now has filled 94% of the available space. Our system administrators gave us a new device which we could mount at an additional mount point. How can we then redirect the index to utilize the new "partition"? Since this is Linux, there is a possibility of stopping Splunk, copying the data over to the new device, then swapping mount points so that Splunk continues to work in the bigger space. I'm just wondering whether this is the optimal approach. ... View more