The cluster manager manages the cluster as a whole. A multisite cluster is still a single cluster, just split into separate sites. At this moment there is no such functionality as "subordinate manager" or something like that. The manager manages the overall cluster contents - most importantly keeps track of buckets across all cluster and makes sure that both site replication/search factors as well as overall replication/search factors are met. And to do so it must have the overall picture of the whole cluster and buckets contained within every single indexer. You can have a fail-over cluster manager, possibly located physically in a different site (but still in terms of Splunk site-affinity it's a "site-less" component). But still at any given point in time you have one manager managing whole of the cluster. ... View more

OK. If you uploaded the file via GUI, as described in that article, it should be _somewhere_. Question is what sourcetype did you give it and whether it got properly timestamped and such. Since it's - as I understand - your small testing installation, verify where your events are. | tstats min(_time) as earliest max(_time) as latest count where index=* by index source sourcetype | convert ctime(earliest) as earliest ctime(latest) as latest Run this search over "all time" time range. ... View more

That's a different parameter. This one just raises the queue size limit but the thruput limit still caps the speed of outgoing events. Verify the output of splunk btool limits list thruput | grep maxKbps The default value of 256 is relatively low and might cause bottleneck. ... View more

It doesn't work that way. Splunk doesn't simply process any files you throw into its directory. See the introduction here: https://docs.splunk.com/Documentation/Splunk/9.0.4/Data/WhatSplunkcanmonitor ... View more

Are you sure you're not hitting the default (relatively low) thruput limit? ... View more

That's one thing. Another thing is that obviously you won't be able to manipulate features that are not enabled in free version (scheduled searches, forwarder management, clustering...). If you have a valid use case consider applying for a dev or dev/test license (read terms for those license types and see if you fit any of those). ... View more

To verify if something exists in a lookup it's usually better to just do the lookup and verify the results either with something like <...something...> | lookup lookup.csv lookupfield OUTPUT lookupfield AS outputfield | eval found=if(outputfield==lokupfield,1,0) Or even <...something...> | lookup lookup.csv lookupfield OUTPUT lookupfield (In this case the field values which are not in the lookup will get nulled-out). The lookup-based approach is more effective compared to inputlookup and stats because lookup is a distributable streaming command and you can still continue processing your search pipeline on the indexers whereas stats will consolidate the results and move processing to SH. ... View more

The inputlookup command is for something completely different - it returns the contents of your lookup file. If you want to perform a lookup, you just need to use... yes, the lookup command! So your original idea was a good one, just with a wrong command index=whatever | lookup employee.csv | [...] ... View more

Did you check your logs for errors? Did you provide Splunk support with the correct external IP of your SH? Did you verify the network-level connectivity? ... View more

One possible solution is of course the one shown by @isoutamo but there is more to it than meets the eye. There are two different kinds of events here so it would probably be best to define two separate definitions for field extractions. Or even two separate sourcetypes. As a side note - that application log seems to be overly complicated - I'd try to work with the app team to make them log in a less nested and easier parseable format. ... View more

From the security point of view - probably the only acceptable solution would be to drop the index altogether and re-ingest the data from scratch - this time properly with sensitive data masked/removed. That of course costs your license usage. You could try to fiddle with masking and collecting to another index but again - collect with a sourcetype other than stash incurs license usage. As @isoutamo already pointed out, the delete command marks the events as unsearchable but they are still contained within the buckets so they might be readable by examining raw buckets content which is probably not OK by your compliance dept. Any form of search-time manipulation within splunk itself is indeed bypassable relatively easy and it can be seen only as a way to hide some event parts for making work with some parts of data more convenient but not as a security control (at least not an effective one). ... View more

If I understand you correctly, you'd like to deduplicate events on ingest - either not ingest an event if there is already one with the same value of a field called ID or overwrite previous values of such field. Well, that's not possible with native splunk functionalities. 1. Splunk ingestion process works one event at a time. 2. Splunk ingestion process works "one-way" - you can't "check what's already in the index". Remember that parsing can be performed way, way before the event even reaches the indexers (and different events from the same source can be processed on different components). Also, you don't have access to search-time extracted values during the ingestion process. 3. There is no "overwriting" in Splunk. So if it's really essential for you that you don't ingest duplicated ID's, you need to design your own ingestion process that will keep your events deduplicated (but for that you'd need some buffer window which will increase latency). ... View more

No, server as such should not crash. Why would it? That's what the OS-level resource management is for 😉 Anyway, how many and what kinds of inputs do you have on this box? Are you hitting any limits? (like throttling outputs and building up queues on the forwarder side) What are the server's specs? (RAM/CPU). Is it busy otherwise? ... View more

It looks more or less like a normal shutdown but there is one interesting thing. 04-16-2023 09:23:40.524 +0300 WARN  TailReader [5296 tailreader0] - Could not send data to output queue (parsingQueue), retrying...  This one. It suggests (but I'm just shooting blindly here mostly) that there might be some overuse of memory and blocked queues and so on. But again - is there any crash log in forwarder's log directory? ... View more

https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/SHCarchitecture ... View more

Well... there is another thing to consider with inputs - why would you want to run your inputs on the search-head? You typically set up a HF for this (for scripted input even a UF should do). And why, oh why, would you distribute such input to all SHC members??? That's counterintuitive - first you want to create several separate instances of your input, then you want to disable almost all of them. Seems pointless. ... View more

As I said before, you probably can use strptime on that field to parse out the timestamp but in this particular case I think it would be even worse-performing. ... View more

Something like stats  values(*) as * | where DCOEtransfercount=1 AND NDNCopycount=1 as the alert condition. Might need further tweaking if you expect multiple result rows in your original results. ... View more

OK. So your options are: 1. Rework your search so that the results are aggregated into single event so that you can compare different fields within a single event or 2. Use something like |stats values in your condition search to "compress" the results and then compare aggregated fields. ... View more