My Server has a Intel i3-3220 3.30GHz CPU and 32 GB RAM where in average I see CPU with 75% usage and 60% of RAM usage ((SPLUNK and other services) HEALTH.LOG entries(one timeslot of today) 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - product="splunkd" color=green node_type=product node_path=splunkd 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="File Monitor Input" color=green node_type=category node_path=splunkd.file_monitor_input 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="BatchReader-0" color=green due_to_stanza="feature:batchreader" node_type=feature node_path=splunkd.file_monitor_input.batchreader-0 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="TailReader-0" color=green due_to_stanza="feature:tailreader" node_type=feature node_path=splunkd.file_monitor_input.tailreader-0 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="Index Processor" color=green node_type=category node_path=splunkd.index_processor 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="Buckets" color=green due_to_stanza="feature:buckets" node_type=feature node_path=splunkd.index_processor.buckets 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="Disk Space" color=green due_to_stanza="feature:disk_space" node_type=feature node_path=splunkd.index_processor.disk_space 12-31-2018 12:24:21.344 +0100 INFO PeriodicHealthReporter - feature="Index Optimization" color=green due_to_stanza="feature:splunkoptimize_processes" node_type=feature node_path=splunkd.index_processor.index_optimization 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - product="splunkd" color=green node_type=product node_path=splunkd 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="File Monitor Input" color=green node_type=category node_path=splunkd.file_monitor_input 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="BatchReader-0" color=green due_to_stanza="feature:batchreader" node_type=feature node_path=splunkd.file_monitor_input.batchreader-0 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="TailReader-0" color=green due_to_stanza="feature:tailreader" node_type=feature node_path=splunkd.file_monitor_input.tailreader-0 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="Index Processor" color=green node_type=category node_path=splunkd.index_processor 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="Buckets" color=green due_to_stanza="feature:buckets" node_type=feature node_path=splunkd.index_processor.buckets 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="Disk Space" color=green due_to_stanza="feature:disk_space" node_type=feature node_path=splunkd.index_processor.disk_space 12-31-2018 12:24:51.346 +0100 INFO PeriodicHealthReporter - feature="Index Optimization" color=green due_to_stanza="feature:splunkoptimize_processes" node_type=feature node_path=splunkd.index_processor.index_optimization 12-31-2018 12:25:21.335 +0100 INFO PeriodicHealthReporter - product="splunkd" color=green node_type=product node_path=splunkd 12-31-2018 12:25:21.335 +0100 INFO PeriodicHealthReporter - feature="File Monitor Input" color=green node_type=category node_path=splunkd.file_monitor_input 12-31-2018 12:25:21.335 +0100 INFO PeriodicHealthReporter - feature="BatchReader-0" color=green due_to_stanza="feature:batchreader" node_type=feature node_path=splunkd.file_monitor_input.batchreader-0 12-31-2018 12:25:21.335 +0100 INFO PeriodicHealthReporter - feature="TailReader-0" color=green due_to_stanza="feature:tailreader" node_type=feature node_path=splunkd.file_monitor_input.tailreader-0 12-31-2018 12:25:21.336 +0100 INFO PeriodicHealthReporter - feature="Index Processor" color=green node_type=category node_path=splunkd.index_processor 12-31-2018 12:25:21.336 +0100 INFO PeriodicHealthReporter - feature="Buckets" color=green due_to_stanza="feature:buckets" node_type=feature node_path=splunkd.index_processor.buckets 12-31-2018 12:25:21.336 +0100 INFO PeriodicHealthReporter - feature="Disk Space" color=green due_to_stanza="feature:disk_space" node_type=feature node_path=splunkd.index_processor.disk_space 12-31-2018 12:25:21.336 +0100 INFO PeriodicHealthReporter - feature="Index Optimization" color=green due_to_stanza="feature:splunkoptimize_processes" node_type=feature node_path=splunkd.index_processor.index_optimization SPLUNK_APP_INFRA.LOG Fetching AWS CloudWatch inputs... 2018-12-21 16:51:57,474 - pid:22664 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 16:51:57,474 - pid:22664 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 16:52:12,667 - pid:22664 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 16:52:12,667 - pid:22664 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:01:56,835 - pid:32484 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:01:56,835 - pid:32484 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:02:07,933 - pid:32484 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:02:07,933 - pid:32484 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:11:56,812 - pid:32660 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:11:56,812 - pid:32660 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:12:08,444 - pid:32660 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:12:08,444 - pid:32660 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:21:55,719 - pid:25808 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:21:55,719 - pid:25808 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:22:05,819 - pid:25808 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:22:05,819 - pid:25808 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:31:56,473 - pid:29756 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:31:56,473 - pid:29756 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:32:07,592 - pid:29756 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:32:07,592 - pid:29756 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:41:56,369 - pid:32512 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:41:56,369 - pid:32512 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:42:08,453 - pid:32512 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:42:08,453 - pid:32512 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 17:51:56,371 - pid:31816 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 17:51:56,371 - pid:31816 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:52:06,469 - pid:31816 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 17:52:06,469 - pid:31816 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:01:56,411 - pid:12408 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:01:56,411 - pid:12408 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:02:07,584 - pid:12408 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:02:07,584 - pid:12408 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:11:56,698 - pid:27228 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:11:56,698 - pid:27228 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:12:08,270 - pid:27228 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:12:08,270 - pid:27228 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:21:57,053 - pid:30736 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:21:57,053 - pid:30736 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:22:07,834 - pid:30736 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:22:07,834 - pid:30736 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:31:56,015 - pid:29452 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:31:56,015 - pid:29452 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:32:08,131 - pid:29452 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:32:08,131 - pid:29452 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:41:56,190 - pid:32600 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:41:56,190 - pid:32600 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:42:07,290 - pid:32600 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:42:07,290 - pid:32600 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 18:51:56,723 - pid:25056 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 18:51:56,723 - pid:25056 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:52:06,296 - pid:25056 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 18:52:06,296 - pid:25056 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 19:01:56,092 - pid:32748 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 19:01:56,092 - pid:32748 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:02:07,686 - pid:32748 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:02:07,686 - pid:32748 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 19:11:56,273 - pid:6972 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 19:11:56,273 - pid:6972 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:12:06,931 - pid:6972 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:12:06,931 - pid:6972 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 19:21:56,549 - pid:9096 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 19:21:56,549 - pid:9096 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:22:07,259 - pid:9096 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:22:07,259 - pid:9096 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 19:31:56,770 - pid:31680 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 19:31:56,770 - pid:31680 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:32:08,361 - pid:31680 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... 2018-12-21 19:32:08,361 - pid:31680 INFO aws_input_restarter:79 - No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... 2018-12-21 19:41:56,397 - pid:7228 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs... 2018-12-21 19:41:56,397 - pid:7228 INFO aws_input_restarter:73 - Fetching AWS CloudWatch inputs... *No AWS CloudWatch inputs found, exiting... No AWS CloudWatch inputs found, exiting... Fetching AWS CloudWatch inputs... Fetching AWS CloudWatch inputs…* (The bold marked entries are reapiting several times (did not count how many times) ... View more

Apologize for not giving Closer Details. Yes, the IP address 192.168.110.15 is Splunk Insights for Infrastructure server. Following entries from today on a specific timeslot I can see: 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - product="splunkd" color=green node_type=product node_path=splunkd 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="Data Forwarding" color=green node_type=category node_path=splunkd.data_forwarding 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="Splunk-2-Splunk Forwarding" color=green node_type=category node_path=splunkd.data_forwarding.splunk-2-splunk_forwarding 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="TCPOutAutoLB-0" color=green due_to_stanza="feature:s2s_autolb" node_type=feature node_path=splunkd.data_forwarding.splunk-2-splunk_forwarding.tcpoutautolb-0 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="File Monitor Input" color=green node_type=category node_path=splunkd.file_monitor_input 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="BatchReader-0" color=green due_to_stanza="feature:batchreader" node_type=feature node_path=splunkd.file_monitor_input.batchreader-0 12-30-2018 16:29:11.655 +0100 INFO PeriodicHealthReporter - feature="TailReader-0" color=green due_to_stanza="feature:tailreader" node_type=feature node_path=splunkd.file_monitor_input.tailreader-0 ... View more

HI, I believe it would make sense to see the whole picture and therefore I tried to attach the splunkd.log file. Unfortunately it says that I do not have enough "karma points"? Following entries from yesterday I could found: 12-27-2018 13:20:48.848 +0100 INFO TcpOutputProc - Connected to idx=192.168.110.15:9997, pset=0, reuse=0. 12-27-2018 13:20:47.541 +0100 INFO TcpOutputProc - Initializing with fwdtype=lwf 12-27-2018 13:20:47.588 +0100 INFO TcpOutputProc - found Whitelist forwardedindex.0.whitelist , RE : .* 12-27-2018 13:20:47.588 +0100 INFO TcpOutputProc - found Blacklist forwardedindex.1.blacklist , RE : _.* 12-27-2018 13:20:47.588 +0100 INFO TcpOutputProc - found Whitelist forwardedindex.2.whitelist , RE : (_audit|_introspection|_internal|_telemetry) 12-27-2018 13:20:47.588 +0100 INFO TcpOutputProc - Initializing connection for non-ssl forwarding to splunk.xy.com:9997 12-27-2018 13:20:47.588 +0100 INFO TcpOutputProc - tcpout group default-autolb-group using Auto load balanced forwarding 12-27-2018 13:20:47.619 +0100 INFO TcpOutputProc - Group default-autolb-group initialized with maxQueueSize=512000 in bytes. 12-27-2018 12:53:37.409 +0100 INFO TcpOutputProc - Shutting down auto load balanced connection strategy 12-27-2018 12:53:37.456 +0100 INFO TcpOutputProc - Auto load balanced connection strategy shutdown finished 12-27-2018 12:53:37.456 +0100 INFO TcpOutputProc - Received shutdown control key. 12-27-2018 12:53:36.904 +0100 INFO TcpOutputProc - begin to shut down auto load balanced connection strategy ... View more

Hi, appreciate your feedback and I can say there wasn't any errors showing up when I ran PS script. SplunkForwarder Service is running and all needed FW ports are open my SPLUNK Insights for Infrastructure Server. A search for "failed" or "error" in the file would give these lines: 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::saveBookMark: Failed to update Windows Event Log bookmark, channel='Application' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::~WinEventLogChannel: Failed to checkpoint for channel='Application' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::saveBookMark: Failed to update Windows Event Log bookmark, channel='Security' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::~WinEventLogChannel: Failed to checkpoint for channel='Security' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::saveBookMark: Failed to update Windows Event Log bookmark, channel='Setup' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::~WinEventLogChannel: Failed to checkpoint for channel='Setup' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::saveBookMark: Failed to update Windows Event Log bookmark, channel='System' 12-27-2018 12:53:38.112 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" ERROR splunk-winevtlog - WinEventLogChannel::~WinEventLogChannel: Failed to checkpoint for channel='System' 12-27-2018 13:21:14.663 +0100 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventMon::configure: Failed to find Event Log with channel name='Forwarded Events' Thanks again for your support, Markus ... View more