Per the DOCS, here: Install the Splunk Add-on for Windows - Splunk Documentation and for metric here: https://docs.splunk.com/Documentation/AddOns/released/Windows/Configuration#Collect_perfmon_data_and_wmi:uptime_data_in_metric_index You should ensure you have a metrics index defined, and install it accordingly at every layer to ensure you're getting the data you need.  ... View more

We had this issue with some of our devices for syslog data, the work around is to use a syslog server. If you are comfortable with Linux, then standup a server with rsyslog, do the appropriate configs and then put a UF on the host and have it monitor the log folder/files, etc. ... View more

This may be a relevant source for additional troubleshooting: Solved: What's the best way to get Windows Perfmon data in... - Splunk Community ... View more

@mooree  You write: " All other logs and events are getting through fine.  " these are  (other  - non-metric) logs from that 2022 server? ... View more

I am not familiar with Splunk on Docker, so I don't have any experience that will be useful here.  Some refs you may find useful:  Architecture | docker-splunk Navigation | docker-splunk Forwarding data into indexer - Splunk Community (Similar question) ... View more

What do you mean by "Security Lockdown"? Are there any local host firewall settings that are active on that server? ... View more

I suppose I'll ask: did you verify network connectivity between the host (with I presume a UF) and the HFs? And the HFs and the Indexing peers? Makig sure there are no issues with switches or firewalls (opening needed ports etc.) ... View more

Hi Manjunatha, Are you having a Splunk issue? Otherwise, If this is an issue just about a (Cisco) switch, that is out of scope for this forum, and you should check with Cisco support. ... View more

This was an issue I struggled with a bit at first, and while the Splunk team is very excellent, there own perspective is not always intuitive with respect to naming and function. Splunk DOCs describe the following about the deployment server, in particular the deployment clients have a defined deployment server that manages the configurations that are pushed out to it see the following: Plan a deployment - Splunk Documentation " Deployment server and clusters You cannot use the deployment server to update indexer cluster peer nodes or search head cluster members. Indexer clusters Do not use deployment server or forwarder management to manage configuration files across peer nodes (indexers) in an indexer cluster. Instead, use the configuration bundle method. You can, however, use the deployment server to distribute updates to the manager node, which then uses the configuration bundle method to distribute them to the peer nodes. See "Update common peer configurations" in the Managing Indexers and Clusters of Indexers manual. Search head clusters Do not use deployment server to update search head cluster members. The deployment server is not supported as a means to distribute configurations or apps to cluster members. To distribute configurations across the set of members, you must use the search head cluster deployer. See "Use the deployer to distribute apps and configuration updates" in the Distributed Search manual."   The reference for respective configuring is here: Deploying Apps: Use the deployer to distribute apps and configuration updates - Splunk Documentation (see this section: https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/PropagateSHCconfigurationchanges#Deploy_a_configuration_bundle) By contrast, apps and configurations are managed by the deployment server, here: Create deployment apps - Splunk Documentation ... View more

@krutika_ag  Maybe I don't entirely understand your scenario. Is there only one syslog server, or multiple ones? The syslog server, if it is properly configured does not just create duplicate entries. Check your syslog configuration both on the server and the sending nodes. As far as ensuring that the ingestion is unique, add a CRC salt and/or ensure there is a stanza in your inputs.conf that is ignoring older files. There is a relevant discussion here: How to avoid reindexing files after setting crcSal... - Splunk Community inputs.conf - Splunk Documentation ... View more

@tuts  As with most things Splunk, the specifics will depend on the details of your environment and chosen deployment. In the question you asked, the Splunk DOCs are sufficient to give you a general idea, but without more details it is harder to give more specific advice. If you're on-prem, then this is another example: Monitor files and directories with inputs.conf - Splunk Documentation Syslog data is sent over port 514, though it can also be configured to transmit and receive on another port and it is usually UDP, which means that they data is best effort but there is no transport guarantee, and so potentially you will lose logs.  Further, depend on your network environment and configuration, you will need to account for any switches and firewalls to ensure that they traffic is being transmitted and received between the source and the syslog receiver, and then the traffic between the syslog server and Splunk (if you have not already configured Splunk to directly receive syslog traffic (and this changes depending on whether this is a linux or windows environment.) In the hint that Giuseppe gave above, RSYSLOG is a linux native solution that can be set-up to receive syslog data from remote hosts, and configured to deposit them to specific folders that can then be monitored by a UF agent on the syslog server host.  ... View more

The Rickest Dill around! Thank you! ... View more

Hi guys, Thanks for responding, and apologies for the long delay. There is an error for one of the multisite variables not being configured properly and then a bunch of subsequent errors that the server cannot reach to the CM. I will also add that this is a HA multisite config and idk how this impacts how the needed cluster stanza variables that need to be configured. ... View more

 Here are some useful references: Get data from TCP and UDP ports - Splunk Documentation Create custom indexes - Splunk Documentation Note the section in the first link, copied below, where it says that you have to define the inputs stanza attributes so that the data ingested is properly indexed. For the second link, the "Create Event Indexes" section might be orienting for you. ================== Configure a UDP network input This type of input stanza is similar to the TCP type, except that it listens on a UDP network port. If you provide <remote server>, the port that you specify only accepts data from that host. If you don't specify anything for <remote server>, the port accepts data that comes from any host. [udp://<remote server>:<port>] <attrbute1> = <val1> <attrbute2> = <val2> ... The following settings control how the Splunk platform stores the data: Setting Description Default host = <string> Sets the host field to a static value for this stanza. Also sets the host key initial value. Splunk Cloud Platform uses this key during parsing and indexing, in particular to set the host field. It also uses the host field at search time. The <string> is prepended with host::. The IP address or fully-qualified domain name of the host where the data originated. index = <string> Sets the index where Splunk Cloud Platform stores events from this input. The <string> is prepended with index::. main or whatever you set the default index to sourcetype = <string> Sets the sourcetype field for events from this input. Also declares the source type for this data, as opposed to letting Splunk Cloud Platform determine it. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing. Sets the sourcetype key initial value. Splunk Cloud Platform uses the key during parsing and indexing, in particular to set the source type field during indexing. It also uses the source type field that it used at search time. The <string> is prepended with sourcetype::. Splunk Cloud Platform picks a source type based on various aspects of the data. There is no hard-coded default. source = <string> Sets the source field for events from this input. The <string> is prepended with source::. Do not override the source key unless absolutely necessary. The input layer provides a more accurate string to aid in problem analysis and investigation by recording the file from which the data is retrieved. Consider use of source types, tagging, and search wildcards before overriding this value. The input file path. indexQueue Sets where the input processor deposits the events that it reads. Set to parsingQueue to apply the props.conf file and other parsing rules to your data. Set to indexQueue to send your data directly into the index. parsingQueue _rcvbuf = <integer> Sets the receive buffer for the UDP port, in bytes. If the value is 0 or negative, Splunk Cloud Platform ignores the value. 1,572,864 unless the value is too large for an OS. In this case, Splunk Cloud Platform halves the value from this default continuously until the buffer size is at an acceptable level. no_priority_stripping = true | false Sets how Splunk Enterprise handles receiving syslog data. If you set this setting to true, Splunk Cloud Platform does not strip the <priority> syslog field from received events. Depending on how you set this setting, Splunk Cloud Platform also sets event timestamps differently. When set to true, Splunk Cloud Platform honors the timestamp as it comes from the source. When set to false, Splunk Enterprise assigns events the local time. false (Splunk Cloud Platform strips <priority>.) no_appending_timestamp = true | false Sets how Splunk Cloud Platform applies timestamps and hosts to events. If you set this setting to true, Splunk Cloud Platform does not append a timestamp and host to received events. Do not configure this setting if you want to append timestamp and host to received events. false (Splunk Cloud Platform appends timestamps and hosts to events) @jmrubio ... View more

Also, make sure that any tagging, eventtypes, and macros are also properly parallel.  Data models can be tricky, there is alot of sometimes subtle things that need to be in place and configured correctly. About tags and aliases - Splunk Documentation Tag event types - Splunk Documentation About event types - Splunk Documentation Use search macros in searches - Splunk Documentation ... View more

Hi man03359, I'd say nothing wrong with being a noob, we all are at some point, continuously. To your question, while perhaps not quite directly responsive: you might want to create tags (and review the exisitng ones) for certain types of data that you know lies in different indexes and sourcetypes. This can be a way of creating a searchable correlation that is properly time-indexed; and then you can pivot to the specific index/sourcetype. Some explanation on tagging/eventypes here:  what is the basic difference between tags and even... - Splunk Community About tags and aliases - Splunk Documentation About event types - Splunk Documentation ... View more

Hi guys, Great discussion, it is both interesting and insightful to get to see and "listen in on" experts having both problems and being willing to do so publicly. Thank you.   Cheers, ... View more

Ismo, Appreciate the response.     ... View more

Hi Delmah, You may find this reference helpful: https://docs.splunk.com/Documentation/AddOns/released/VMW/Installationoverview ... View more