Actually you have to adapt the local.meta from the search app. It has been tested with 8.0.3 1. edit local.meta vi /opt/splunk/etc/apps/search/metadata/local.meta 2. add the following line: [views/job_manager] access = read : [ admin ], write : [ admin ] export = system owner = admin   ... View more

Hi Why dont you filter the events with the CSV with the inputlookup command? In this way you only get the events where assigned_to contains only those people of your list. index="one" sourcetype="generated" Number=* status=* group=* [ | inputlookup test.csv | field name | rename name as assigned_to ] Would that help? ... View more

Hi @bobryant You said that you put the inputs.conf and props.conf on the UF. is that correct? If yes it's explainable why the linebreaking doesnt work. your props.conf should be located on the first Splunk Enterprise Instance. This instance is responsible for the parsing and will do the linebreaking and timestamp extraction. This can be either a HF or a Indexer. I just noticed that you use the BREAK_ONLY_BEFORE parameters. I would exchange it with LINE_BREAKER and SHOULD_LINEMERGE=false . This way you get more performance out of Splunk and is also best practice doing that. The config could look like this: [appx:ipro] LINE_BREAKER = ([\r\n]+)\d{2}:\d{2}:\d{2}\.\d{3} MAX_TIMESTAMP_LOOKAHEAD = 30 NO_BINARY_CHECK = true SHOULD_LINEMERGE = false ... View more

Hi @jiaqya What you are looking for is Search Affinity. Have a look at this documentation: https://docs.splunk.com/Documentation/Splunk/7.2.4/Indexer/Multisitesearchaffinity The Replication of a bucket is fast, but you need the network bandwidth for it. A default bucket size is 750 MB ( maxDataSize = auto ) and 10GB if you use maxDataSize = auto_high_volume ... View more

Hi @Deepz2612 I have tried this regex for extracting the field messsage. (it worked with regex101.com) \"(?<message>\w+\s\S+)\? ` wit rex it should look like this: rex field=api_call "\"(?<message>\w+\s\S+)\?" ... View more

Yes, the data will be available on both sites. You can lose one site and you have still all data. The searchfactor is configured with: site_search_factor = origin:1, total:2 So if site1 goes down your Searchheads are still able to search for all data. ... View more

Hi @gedworksplunk I have some thoughts about your configuration. First of all NFS is not suggested by Splunk for using on the Indexers. If possible use local storage or reliable storage subsystem. In your setup you will only backup the replicated buckets, but you will never get the master buckets. So in your backup strategy you miss the original data. (Sitenote: We had some issues with replicated bucket when they were made searchable. Especially with buckets which contained Windows event logs) The backup strategy of Splunk is very simple, which is described here: https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Backupindexeddata Just install a backup agent and implement Splunks recommendation. Another thought regarding Mulitsite Clustering Backup. If you backup all the data from each indexer you probably will backup more data as need. I would backup only the original buckets (db_) and write a blacklist for the replicated buckets (rb_). Why? The cluster will automatically replicate the original bucket if the clustermaster doesnt find any copies ... View more

Hi @jiaqya Lets go through site_replication_factor = origin:1 total:2 This config origin:1 means that one copy of the bucket is on the site where the main bucket is created. total:2 means that 2 copies of the main bucket has to exist. In your setting it means that the other bucket will be stored on site2. site_replication_factor = <comma-separated string> * Only valid for 'mode=master' and is only used if 'multisite=true'. * This specifies the per-site replication policy for any given bucket represented as a comma-separated list of per-site entries. * Currently specified globally and applies to buckets in all indexes. * Each entry is of the form <site-id>:<positive integer> which represents the number of copies to make in the specified site * Valid site-ids include two mandatory keywords and optionally specific site-ids from site1 to site63 * The mandatory keywords are: - origin: Every bucket has a origin site which is the site of the peer that originally created this bucket. The notion of 'origin' makes it possible to specify a policy that spans across multiple sites without having to enumerate it per-site. - total: The total number of copies we want for each bucket. * When a site is the origin, it could potentially match both the origin and a specific site term. In that case, the max of the two is used as the count for that site. * The total must be greater than or equal to sum of all the other counts (including origin). * The difference between total and the sum of all the other counts is distributed across the remaining sites. * Example 1: site_replication_factor = origin:2, total:3 Given a cluster of 3 sites, all indexing data, every site has 2 copies of every bucket ingested in that site and one rawdata copy is put in one of the other 2 sites. * Example 2: site_replication_factor = origin:2, site3:1, total:3 Given a cluster of 3 sites, 2 of them indexing data, every bucket has 2 copies in the origin site and one copy in site3. So site3 has one rawdata copy of buckets ingested in both site1 and site2 and those two sites have 2 copies of their own buckets. * Default: origin:2, total:3 ... View more

Hi @evelenke Your setup sounds quite similar than ours. We collect syslog with rsyslog and put them into a structured folder system like /var/log/rsyslog-splunk/uc////logfile.log We have also several hundreds of files on our servers. First of all make sure that you configure the ulimits as described by @lakshman239 (https://docs.splunk.com/Documentation/Splunk/7.2.4/Installation/Systemrequirements) We also raised the thruput limits (limits.conf): maxKBps = 0 [thruput] maxKBps = <integer> If specified and not zero, this limits the speed through the thruput processor to the specified rate in kilobytes per second. To control the CPU load while indexing, use this to throttle the number of events this indexer processes to the rate (in KBps) you specify. The second config we did was raising the pipelines to 4 (server.conf). parallelIngestionPipelines = 4 parallelIngestionPipelines = <integer> * The number of discrete data ingestion pipeline sets to create for this instance. * A pipeline set handles the processing of data, from receiving streams of events through event processing and writing the events to disk. * An indexer that operates multiple pipeline sets can achieve improved performance with data parsing and disk writing, at the cost of additional CPU cores. * For most installations, the default setting of "1" is optimal. * Use caution when changing this setting. Increasing the CPU usage for data ingestion reduces available CPU cores for other tasks like searching. * NOTE: Enabling multiple ingestion pipelines can change the behavior of some settings in other configuration files. Each ingestion pipeline enforces the limits of the following settings independently: 1. maxKBps (in the limits.conf file) 2. max_fd (in the limits.conf file) 3. maxHotBuckets (in the indexes.conf file) 4. maxHotSpanSecs (in the indexes.conf file) * Default: 1 ... View more

We had similar issues with a unix forwarder. I has to monitor over a million of files. It stopped sending logfiles and had high CPU consumption. I assume you are struggling with the same issue as we did. The UF doesnt like haveing lots of open files like you configured with alwaysOpenfile=1 Even if you get rid of this option I doubt that the UF will work. Try using the batch:// option. It reads and deletes the file. if you dont want to delete the file on your server I suggest, that you write a script in which the files are copied to a temporary directory. In the end you will have configuration like this (inputs.conf): [batch://c:\FirstFolder\Log\service\System\20*\PROXY\...] disabled = false index=my_index sourcetype=proxy move_policy = sinkhole ... View more

It think the issue lies in the stanza definition. I had very bad experience with whitespaces in it. Just try [alert] as stanza ... View more

It might be possible that something is odd with the indexing or parsing queue on the Splunk Cloud Instance. And the problem on your Windows Servers are just sideeffects. But this would have effects on other servers too. Have a look at the Management Console -> Indexing - Indexing Performance: Deployment . At the Indexing Performance by Instance Panel you can see if you have issues with any of the queues. ... View more

According the transforms.conf documentation: _meta : A space-separated list of metadata for an event. Honestly I dont get your question, but I advice you to visit the documentation for transforms.conf. It has a lot of examples, which can help you. If not feel free to clarify your question. ... View more

Thats strange. maybe its an issue with the universalforwarder. If you only have one subdirectory you could you * instead of ... No issue with working with whitelists as long as the sourcetype is the same. Otherwise different monitoring stanzas would be great. ... View more

Hey. The config looks fine to me, even though I would change the wildcard to the actual logfilename(s) like mylogfile*.log or so. You can never be sure that someone uses your directory as temporary storage. those files would be ingested too. You dont want that. Do you have any logs from the universalfowarder. Maybe the monitored files are to small so that you have to use saltcrc. or maybe you have a permission problem, even though i think that might not be the case on windows machines. ... View more

Hey The config is the same as described above. You dont have to config anything regarding your Windows TA. You just have to copy on the HF and it works. if you the whole Windows-TA 1:1 on the Heavyforwarder, be aware that only two configfiles are needed ( props,conf and transforms.conf ) In our environment I make a new App (like Windows-TA-hf, depends on your naming convention) and put only the needed files into this app. We manually define Linebreaking and the Timestamp because of Splunks Best Practice and of performance reason. ... View more

no it doesn't. The universalf forwarder handles the input phase. SHOULD_LINEMERGE and TIME_FORMAT are part of the parsing phase which does Heavyforwarders or Indexers ... View more

On the forwarder you need the following configuration: [<sourcetype>] disabled = false INDEXED_EXTRACTIONS = CSV FIELD_NAMES = <comma separated field names> Indexers or Heavyforwarders, which are responsible for the parsing phase need this configuration: [<sourcetype>] disabled = false SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = <time stamp field> TIME_FORMAT = <format of time field> ... View more

I just tried it with a few sample logs. See the updated config from my original comment. ... View more

could you please add some example logs? thx ... View more