Getting Data In

Universal Forwarders are phoning home but the indexes are not populating

packet_hunter
Contributor

So while I was out, some Windows config changes were pushed to some Windows servers that had fully deployed UFs with deployed-apps. Prior to these windows changes, the servers were sending wineventlogs via UFs to the indexers without issue. Now the UFs are phoning home but I am not able to see any data since the time the windows changes took place. In fact, since the changes the indexes do not show when I run the following search AFTER the time of the changes,

|tstats values(sourcetype) WHERE index=* by index

The indexes do show up when I run the search BEFORE the time changes were made, which makes sense.
It appears all windows related indexes are down, any advice on where to start troubleshooting?

Thank you

0 Karma
1 Solution

pradeepkumarg
Influencer

Do you get the internal logs from those UFs? That's your starting point
Is the outputs intact on the UFs ?
If you get the internal logs.. check for any errors on splunkd logs.

View solution in original post

0 Karma

pradeepkumarg
Influencer

Do you get the internal logs from those UFs? That's your starting point
Is the outputs intact on the UFs ?
If you get the internal logs.. check for any errors on splunkd logs.

View solution in original post

0 Karma

packet_hunter
Contributor

Thank you for the reply.
The original architect of the splunk UFs confirmed that the two original deployment apps for the UFs were disabled and not deployed to the UFs. Therefore the UFs did not have inputs and outputs.
Your suggestion was correct.

0 Karma

packet_hunter
Contributor

Please convert your comment to an answer thank you.

0 Karma

packet_hunter
Contributor

also do you have a link reference for getting the UF internal logs ? I did not have to go down that path this time but it would be good to know. Thank you

lfedak_splunk
Splunk Employee
Splunk Employee

I converted the answer so you can now accept it! 🙂

0 Karma

pradeepkumarg
Influencer

Internal logs are by default forwarded to the indexers provided you have the outputs set up. you can search like below

index=_internal host=myhostname

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!