I have 2 sites configured with a multisite cluster. Site 1: Indexer, manager, independent search head. Site 1: Indexer, independent search head. I am looking for a way to have all user's searches and lookups synchronized to both search heads, and it looks like a search head cluster is the way to go. First question: I am having a little trouble grasping the cluster+affinity and how it applies to a search cluster. My bandwidth between sites is quite low, and i do NOT want a site1 searchhead searching site2 and vice versa. If I leave affinity active and run a search at site1, does the search take place ONLY on the site1 searchhead against the site1 peer? Or is the search distributed across the 2 sites, even though it is only using the site1 index as the index to search? Second question: I have no need for a 3rd search head. I don't have dedicated hardware, I don't need more resources, etc. But, I know that the search cluster needs 3 searchheads. Can I add a second searchhead at site1 to be the third, but leave it severely underpowered? A small dual core VM whose only purpose is to provide a 3rd member. If I do this and perform a search at site1, will it attempt to search using both the strong and weak search head and slow me down? What it comes down to is that I do not need the full "cluster", I just need to keep my user data replicated. It seems that the cluster is the best way to accomplish this, even though it adds complexity. ... View more

Our splunk system has the potential to grow significantly in the near future, so a veeam backup of the indexer VM will not be practical for long term storage (5 years). My plan: 2 indexers in a multisite cluster. 100% of my data will exist at each site. The hot +warm buckets + OS will be on the primary partition, backed up by veeam with VSS. The cold/frozen will be archived off to long term. Warm/Cold/Frozen will be something like 6 months/1.5 year/5year. But, I don't want to be completely dependent on synchronization of my indexes between the 2 sites, and data corruption/loss will involve some severe implications for us. What I would like to do is have a hot bucket roll off to both warm AND frozen at the same time. That way, my long-term storage is using the smaller buckets, I don't have to worry about waiting for things to age off to frozen, and I have a nice safe copy of my frozen data. I see that buckets can go to different locations, but is it possible to route a bucket to 2 locations/statuses simultaneously? ... View more

I have 2 locations, and not a ton of resources. Multisite clustering took too much -- it seems like I need at least 3 indexers (or maybe it was 2 per site). But, I only have 2 indexers, so I decided a multisite cluster was more then I needed. Instead, I set up a basic index cluster that I was hoping to have span multiple locations. Main goal = data safety. 2 copies of active splunk indexes, plus backups at each location looks to be exactly what I need. But, my pipe between sites is pretty limited. Ideally, my search head would be tied to a specific indexer, so I am not trying to pull data across sites. I looked at affinity (but that is multisite only) and distributed search (but that is non-cluster only). Is it possible to restrict my SearchHead1 to only search Indexer1? ... View more

I have a regular expression that works on part of my data. Given the log entry: pam_vas: Authentication <succeeded> for <active directory> user: <bobtheperson> account: <bobtheperson@com.com> reason: <N/A> Access cont(upn): <bob> i can use the regular expression: [\>\:]*\s+(.*?)\:?\s\<(.+?)\> and get the result I am looking for. (http://regexr.com/3fatg) Authentication = succeeded for = active directory user = bobtheperson account = bobtheperson@com.com reason = N/A Access cont(upn) = bob Unfortunately, when I was building this regular expression, I was ignoring a vital part of the log -- the first part. The log actually looks like this: Feb 16 20:04:37 hostname su[1111]: [id 123456 auth.info] pam_vas: Authentication <succeeded> for <active directory> user: <bobtheperson> account: <bobtheperson@com.com> reason: <N/A> Access cont(upn): <bob> My extraction no longer works right -- it is thrown off by the first part. (http://regexr.com/3fbod) How would I exclude the beginning information from this log file? **Feb 16 20:04:37 hostname su[1111]: [id 123456 auth.info]** pam_vas: Authentication <succeeded> for <active directory> user: <bobtheperson> account: <bobtheperson@com.com> reason: <N/A> Access cont(upn): <bob> I think I need to start my search after the last occurrence of a ] (right before pam_vas) but I cant figure out how to exclude that. ... View more

I am attempting to parse a solaris log file into key/value pairs. The log is: pam_vas: Authentication <succeeded> for <active directory> user: <bobtheperson> account: <bobtheperson@com.com> reason: <N/A> Access cont(upn): <bob> The result I am looking for will be: Authentication = succeeded for = active directory user = bobtheperson account = bobtheperson@com.com reason = N/A Access cont(upn) = bob My testing shows that the expression [\>\:]*\s+(.*?)\:?\s\<(.+?)\> should work. http://regexr.com/3fatg In Splunk, i put this regular expression into a search that returned the log in question. mysearch | rex field=_raw "[\>\:]*\s+(.*?)\:?\s\<(.+?)\>" It returned an error: Error in rex command. The regex does not extract anything. It should specify at least one named group. Can you help me turn this into an actual key/value pair list of results? ... View more