The bucket remained in thaweddb ... View more

If there are no capabilities to restrict individual users from writing to any index, then the issue is definitely not fixed. ... View more

Mate, you're a lifesaver on both counts. Thank you very much! ... View more

Thanks for your suggestions! Unfortunately the top example using a macro doesn't work -- that literally searches the string "jobsearch" The bottom one I can't seem to work into my full search. eventtype=my_eventtype source="*logfile.log" | rex "[Tt]he job '(?[^']+)'" [ | gentimes start=-1 | eval job="email alerts" | makemv job | mvexpand job | table job ] | transaction host job session startswith="Started running the job" endswith="has succeeded" | table _time session job duration Gives me an error: Error in 'rex' command: Invalid argument: '(' PS: How do I use that codeblock you've used? I'm not sure what the markup is for it and the code html tag is awful. ... View more

Thanks for your response! Limit the access to the indexes that the user can access via their role, and then take away the capability from that role to do any writes. So this sentence is rather promising to me. Our current setup is as follows: User role is mostly default. As per the roles section under authorisation, it has the following capabilities (none are inherited): accelerate_search change_own_password get_metadata get_typeahead input_file list_inputs output_file pattern_detection request_remote_tok rest_apps_view rest_properties_get rest_properties_set search The user role also has a number of indexes (upwards of 30) configured in both srchIndexesAllowed and srchIndexesDefault (both the same), but there are easily another 20-30 indexes which are not configured for the default user role. The user role can write to those indexes that have been disallowed -- we have tested with a given user who wrote a log entry from an index they could see into an index they were unable to see, using the collect command. So my questions would be, with regard to the first sentence, are: Is there another way of limiting role access to a given index? Which capability grants write access to indexes? What capability can be used to disallow access to the 'collect' command? You can also prevent the index from being written to from the Search Head by updating the local search head indexes.conf to have: This is a potential idea that we can use, but we'd have to adjust it in a way that kind of moots the point of having a search head cluster. If we were to prevent the search head cluster from writing to indexes, then we would have to set up search heads outside of the cluster to run scheduled searches which then write to summary indexes, as we heavily use the summary indexes. This would mean creating a standalone search head which would be able to do the summary index writing -- however this is something we are trying to avoid, as we are trying to make our Splunk setup as resilient as possible in terms of site failure and failovers: every search head on one site should have a counterpart on the other site, ready to take over automagically. Depending on what you are trying to prevent, either Users being malicious, or users making mistakes. Using a deployment server to manage forwarders really cuts down on end users being able to change the configs. We have two deployment servers (active & passive / hot-warm), but most users (apart from on Windows) do not have the user permissions to modify Splunk forwarders or their configs without prior approval and action from the appropriate administrative team. The network ACL is worth another look from our side, as there are some clear subnet divides that we could enforce, but this is too general a solution, unfortunately. ... View more

This is still a bug in 6.3.3. I've had to apply the workaround to fix this issue after not restarting the search head cluster for a week. No reference to this in the known issues however, nor any response from Splunk support. http://docs.splunk.com/Documentation/Splunk/6.3.3/ReleaseNotes/KnownIssues ... View more

Seems to work fine. Thanks a lot mate 🙂 ... View more

Yep -- you're right for this one. Thanks ... View more

I can verify that, of the top list, the following are missing: edit_search_scheduler (doesn't exist in 6.2.3, the current version we're on -- should be moving to 6.3.x in a month or so) edit_token_http edit_win_admon edit_win_eventlogs edit_win_perfmon edit_win_regmon edit_win_wmiconf list_pdfserver list_search_scheduler list_win_localavailablelogs web_debug (doesn't exist in 6.2.3) Of the bottom list, I'm not sure exactly how to get most of these are turned on -- only schedule_rtsearch is appearing -- but I'm sure that a number of these are turned on for admin users. ... View more

Unfortunately not -- hopefully should be soon, though. Thanks for the URL. ... View more

Giving it a go, will report back after a couple of cluster elections. Thanks for the idea! ... View more

That's the one, couldn't find it for the life of me. Cheers!! 🙂 ... View more

http://docs.splunk.com/Documentation/Splunk/6.2.3/Knowledge/ExtractfieldsinteractivelywithIFX ... View more

Unfortunately I was 😞 ... View more

Thanks for the response. In response to the bullet points: Forwarders are all set up using configs from the deployment server. All of these forwarders have been restarted relatively recently (for the ones that are causing the biggest headaches, we restart them once a week) Can't find any documented bug We have independent monitoring which alerts the support teams upon disk issues, so this is within bounds The main issue we seem to get is that for files that aren't written to as often, they can stop being monitored by the forwarders from time to time. This is particularly true of one application, which logs infrequently, but when it logs, the information is rather vital. On the same forwarder, it will have stopped monitoring one log file, but is still monitoring the others in its inputs.conf (same custom Splunk collector, in the same input stanza). There's nothing in the splunkd.log, nor anything I can see in the metrics either. It simply stops monitoring the log and stops reporting that it's monitoring the log. As with the case in the original post, if it gets to this point, it simply stops monitoring for new files. ... View more