Yes those are independent, but as one file is actively read until all event are handled then one big active file don't block the traffic as there are another pipelines available to process other files instead of waiting when that big high volume file has read. ... View more

You need to convert start time to epoch and then sort by it. | eval sTime = strptime(startTime,"%FT%T.%3Q%Z") ... View more

Hi In transaction command reference it Is said Descending chronological order required The transaction command requires that the incoming events be in descending chronological order. Some commands, such as eval, might change the order or time labeling of events. If one of these commands precedes the transaction command, your search returns an error unless you include a sort command in your search. The sortcommand must occur immediately before the transaction command to reorder the search results in descending chronological order. So you must sort those into correct order before you can use successfully transaction command. ... View more

One additional point MC + DS live together. It depends if your admins and persons are same staff or not. As MC is usually targetted only to admins not other persons, you should really think if you want to mix those roles. I know that many cases this is not issue but still something to consider. ... View more

Neither me. I like to say that only reason to cluster DS it that you have so many clients that one DS cannot manage it. Currently official values is probably 25k after that one recommended host cannot server those with reasonable load and response times. If you are just needing HA feature then it's better to automate you DS configurations and also automate build up DS and use only names in any configurations. Then if your primary DS will got broken/unavailable etc., then just build a new one and redeploy configurations to it. Of course this probably enforce all your clients to reload their apps and then restart splunkd. Anyhow I strongly recommend to all, that use DS GUI just to look situation, but never ever use it for configure DS. All configurations should be (or must) in e.g. git and you have way to deploy those as needed into (any) DS. ... View more

If I have understand correctly how splunk UF is doing this and you have only those two high volume logs then (probably) you can try to prioritise those by adding pipelines. I'm not 100% sure this, but my understanding is that in normal situation on files will be read to the end and then UF switch to another. If this is true then those high volume files could have dedicated readers and other shares additional. But remember that you cannot add too many pipelines per node! And as I said I haven't been to test this in real life by myself. There have been cases where I have added some pipelines e.g. in HF (DBX) to get it working correctly. And I expecting that this is working UF too? I don't believe that this is "approved by Splunk" solution 😉 ... View more

As said this hasn't beed recommended way to do before 9.4 as @richgalloway said. But if you have some older versions and those are behind LB then you probably (or at least you should have) this settings in use in your serverclass.conf crossServerChecksum = <boolean> * Determines whether the deployment server ensures consistent app checksums across multiple deployment servers. * A value of "true" means the deployment server generates the same checksum for each app across different deployment servers. This setting is useful if you have multiple deployment servers behind a load balancer. * A value of "false" means each deployment server generates its own checksum independently. * Default: false  If I recall correctly there are conf19? or something presentation how you could configure this in old way. Here is one old post which also could help you if you have that setup https://community.splunk.com/t5/Deployment-Architecture/multiple-deployment-servers-checksum-mismatch-among-instances-of/m-p/167120 But currently if you need several DS then use official way which is defined in those help pages which Rich pointed to you. ... View more

And don't remember to check that someone haven't hardcoded those names + IPs into /etc/hosts 😞 ... View more

You could try this to check how splunk see those roles and if there is some other roles inherited to your role or users. https://community.splunk.com/t5/Splunk-Search/How-to-Generate-Roles-to-Indexes-table/m-p/654874/highlight/true#M226224 ... View more

When you are looking access_combined from .../etc/system/default/props.conf you see that it's using transforms REPORT-access = access-extractions Then when you looking it, you see lot of recursive definitions where those parts are divided. Those are after this comments ######## access-extractions helpers start ######## # make sure to handle escaped quotes (\") inside the URI [access-extractions] # matches access-common or access-combined apache logging formats # Extracts: clientip, clientport, ident, user, req_time, method, uri, root, file, uri_domain, uri_query, version, status, bytes, referer_url, referer_domain, referer_proto, useragent, cookie, other (remaining chars) # Note: referer is misspelled in purpose because that is the "official" spelling for "HTTP referer" REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]](?:\s++"(?<referer>[[bc_domain:referer_]]?+[^"]*+)"(?:\s++[[qstring:useragent]](?:\s++[[qstring:cookie]])?+)?+)?[[all:other]] all those [[<type>:<field name>]] are tokens which get values. Just look deeper which each <type> is, then maybe you need to look again what those contains etc. e.g access-request is  [access-request] # very relaxed regex for extracting fields from the request REGEX = "\s*+[[reqstr:method]]?(?:\s++[[bc_uri]](?:\s++[[reqstr:version]])*)?\s*+"  I contains e.g. bc_uri this is defined as [bc_uri] # backwards compatible uri regex # uri = path optionally followed by query [/this/path/file.js?query=part&other=var] # path = root part followed by file [/root/part/file.part] # Extracts: uri, uri_path, root, file, uri_query, uri_domain (optional if in proxy mode) REGEX = (?<uri>[[bc_domain:uri_]]?+(?<uri_path>[[uri_root]]?[[uri_seg]]*(?<file>[^\s\?/]+)?)(?:\?(?<uri_query>[^\s]*))?) And so on.   ... View more

Maybe you should try to install it and just follow your instructions. If you can’t do it by that way, then update those as you could do perfect installation, update and configuration of it! ... View more

1st here is lis which roles you could combine to same server https://help.splunk.com/en/splunk-enterprise/administer/distributed-deployment-manual/10.2/overview-of-splunk-enterprise-distributed-deployments/components-that-help-to-manage-your-deployment Based on that your combination is ok, if you haven’t have more than 50 clients.  I have seen this kind of situation when node hasn’t have correctly defined host and server names. In those cases it has registered LM wrongly and MC’s queries haven’t match those. You could test this by opening this dashboard’s query and change LM names there. ... View more

I haven’t look the latest ta for Unix and Linux to see what it’s collecting. Maybe this is one of those things and then you should use it. Anyhow in UF introspection index is not collecting all those same values than it’s collecting on full enterprise instance. If you want to collect same kind of data set you need to add it into collections. There are instructions in help/docs how to do it. I can’t remember if those sets are some (probably not) or not, but you could try it. ... View more

Splunk SPL is totally different from SQL in a fundamentally level. The technology and architecture behind those are totally different. This leads to situation where you cannot use SQL directly as a replacement of SPL! There are https://help.splunk.com/en/splunk-enterprise/spl-search-reference/9.2/quick-reference/splunk-spl-for-sql-users for users which are familiar with sql but not with SPL. I’m not sure if this is right thing to do to you, but I still post this here. If you are using splunk 10.2 or newer (yes I know that CMP is still on 10.2) then you can start to look SPL2 as an option too. ... View more

If I recall right there have been discussions about oauth2 with Splunk’s future releases. Maybe you could found something from participating voc.splunk.com? But oauth2 is targeting to authenticate and authorize into splunk searches, not for ingesting data via hec or other ways. Maybe there will be some modular inputs in future which will be use it! But currently e.g. HEC will be using only token based authentication only. ... View more

I think that the big question is “can I lost events?” Then how much I can lost? If the amount is reasonable then follow what @diogofgm said. If you are thinking that you can’t lose any events and you still are using syslog, please think again. Then follow @PickleRick ‘s thinking. In reality using syslog as method to deliver events without any losses it’s really hard almost impossible. I know that @PickleRick will said that it’s impossible 😉 IMHO it’s depends on your architecture and setups, but at least in some cases it’s doable maybe not all cases? ... View more

In officially when you are using free license there are no unlock licenses key. Instead of you have 2 options: wait until it opens, with free it was something like less than 5 license overflow in 30 days if I recall right? Please check actual days from help.splunk.com buy a real license and then you need/can request unlock key from support  install and configure it from scratch (this is really in really dark grey zone as you are already using free license) ... View more

Splunk has created some (or actually quite many) parsing tokens for different http URLs. You should just look those from $SPLUNK_HOME/etc/system/default/props+transfoms.conf files. Basically you could utilize those generic tokenizations in your own definitions too. Those are parsing URLs quite well.  ... View more

Then you need to understand what you are meaning with inside two minutes.  Is this meaning as xx:y1:zz or is this meaning that event has happened within two minute time slot counting ms too? If 1st is enough then bin is correct answer but if it’s 2nd then you need something like stats + range. And try always use first stats instead of *stats as this way you can utilize indexes parallelism (map + reduce) and get better response time and utilize less resources! ... View more

Hi i think that you are looking this https://splunkui.splunk.com? With it you can do almost anything what you ever get. I have seen e.g. Doom inside Splunk UI 😉 ... View more

If I remember right in SPL2 is using different REGEX engine. This means that it’s mandatory to use ?P instead of ? On those REGEX. Probably there was something else too, but you could found those differences from manual. ... View more

If I understand right your issue, the UF is sending to Cribl but not to splunk. Then you basically cannot see anything in splunk side. If you want to see this information in splunk side it needs that at least internal logs are sent to splunk. If true then you can see that uf has connected to Cribl collector. But if they hav3 used Cribl edge then there aren’t any entries in splunkd.logs ... View more

Excellent questions. I cannot recall that there was any confusion when I configure it. All those what it asked works as expected w/o any hassle. But now when I try to go to configuration it just gives empty page for me and safari show just loading on tab. I need to try more time to figure it out. ... View more

Cribl have own slack where you could ask this kind of questions. ... View more