Getting Data In

Some data are not being send to Splunk

szukaczov
Engager

Hi team,

We had some issues with the Splunk forwarder which was not sending data to Splunk. After restart of the service we started to see only part of the logs. 

Logs which we are able to see are from: DNS index

Logs which we cannot see are from MS_AD index and are related to Domain Controller logs. 

 

From the debug log I can see below lines:

 

06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - IndexKey: ms_ad shouldForwardIndex: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22656 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22656 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21272-21272 idx=xxxxx:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21352 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21352 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - IndexKey: ms_ad shouldForwardIndex: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21275-21275 idx=xxxxxxx:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22657 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22657 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21277-21277 idx=xxxxxx:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21353 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21353 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21279-21279 idx=35.234.126.255:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - IndexKey: ms_ad shouldForwardIndex: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22658 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22658 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21281-21281 idx=xxxxx:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21354 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21354 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21283-21283 idx=35.234.126.255:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - IndexKey: ms_ad shouldForwardIndex: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Cache Hit - indexKey: ms_ad shouldForward: 1
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22659 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=22659 on chanID=0 to back of tcp client (tcp output) queue
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Received ACK for : 21285-21285 idx=xxxxx:9997
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21355 on chanId=46 to back of tcp client (tcp output) queue. source:source::WinEventLog:Security|host::xxxxxx|WinEventLog:Security|
06-09-2021 16:54:22.752 +0800 DEBUG TcpOutputProc - Pushed eventId=21355 on chanID=0 to back of tcp client (tcp output) queue

 

Does the above log means that the logs are indexed and will be shown soon in Splunk? 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...