Adding more hosts to searches on SplunkLight

[bybarra94]

I just got into the position I'm in and need some help. We recently added 6 computers to the network, and need to have their data pulled to the Splunk dashboard. How would I go about getting those systems added?

[Richfez]

Well, the good news is you have at least one system already configured to be ingested into Splunk, so that helps immensely.

I'm going to assume you do not have access to whoever set this up initially, because if you did the obvious answer is to ask that person. Also, I'd heartily suggest reading through, and attempting as best you can, the following documentation on inheriting a Splunk installation:
https://docs.splunk.com/Documentation/Splunk/8.0.0/InheritedDeployment/Introduction

If everything's set up simply and in a straightforward manner, things might not be hard. But this isn't always the case - there are myriad ways to set this up, and lots of them can be quite confusing so just stick with it, ask questions, and maybe if you can hop on Splunk's slack channels (google for it, it's an easy process) if you get stuck somewhere. Or ask new questions here in Answers!

Anyway, without further ado, let's see if I can get you started.

What you'll need to do is:
- install the Splunk Universal Forwarder (UF)
- configure it like the other systems that you already have forwarding.

Now, that was a super simplified version. For instance, you may collect data via WMI or by , or you might be using a Deployment Server to deploy configurations.

If you check your existing systems that do come into Splunk, you'll see on them most likely a Universal Forwarder installed. That will be in Program Files for Windows, or (by default) /opt/splunkforwarder in *nix.

First check - do you have a deployment server set up?:
Search on an existing client that sends in data for a file "deploymentclient.conf". It will live in some "local" folder like $splunkhome$/etc/apps/someapplication/local/deploymentclient.conf or $splunkhome$/etc/system/local/deploymentclient.conf. (Where $splunkhome$ is just shorthand for "the folder/directory where the splunk forwarder is installed).
A bit more help on the configuration files can be found here: https://docs.splunk.com/Documentation/Splunk/8.0.0/InheritedDeployment/Confdiscovery
Or, maybe check here to see: https://docs.splunk.com/Documentation/Splunk/8.0.0/InheritedDeployment/MCdiscovery

If you have one, then my suggestion is install the new UF on your new system, and then copy that deploymentclient.conf file into the same place on the new system and restarting splunk (making any folders you need to to add that file, and also confirming permissions and stuff on those). You can read a bit more about this here:
https://docs.splunk.com/Documentation/Forwarder/8.0.1/Forwarder/HowtoforwarddatatoSplunkEnterprise#C...
And there's lots more help available on that, google for it and check out the rest of the Splunk docs pages.

If there is no deploymentclient.conf file, then proceed below:

Finding out how to send new data in:
Like above, look in your various local folders (etc/system/local, or etc/someapp/local) for an "outputs.conf" file (it's a text file, editable by any text editor) to confirm the settings for where to send your data.

During the install of a Windows UF, it should just ask you where to send data and you should be able to figure out the answer from that file (and refer to the docs, linked below)

On *nix it's likely you can just add an output per that doc, or even just copy and paste in the outputs.conf file you have into the same place on your newly installed UF and restart splunk.

At the end of this process, you should now see logs in the index=internal for your new host (and also it'll show up in various other places, like in forwarders and stuff).

Now, on what data to collect:
If you are enabled for being a deployment client, well, hopefully this takes care of itself. But if not, you'll have to adjust some serverclass settings (e.g. "Which apps do we send to which forwarders") and make it look like one of the other deployment client's configurations. https://docs.splunk.com/Documentation/Splunk/8.0.0/Updating/Forwardermanagementoverview#Access_the_f...

If you are not, then you'll have to look at the various "inputs.conf" files on an existing, similar forwarder and ... add those into your new UFs setup. This can be as easy as copying the file around and restarting Splunk.

https://docs.splunk.com/Documentation/Forwarder/8.0.1/Forwarder/Abouttheuniversalforwarder

I know all this sounds like a lot of work - and I won't lie it very well could be. But remember Splunk is a general purpose app and can scale from a single machine reading a single log file with barely even one user, to hundreds of Splunk servers reading a petabyte of data per day serving hundreds or even thousands of users. I think you'll fall far closer to the simple side than the complex, so hopefully this gets you started!

And of course, if you get partway through this process and get stuck again, start up a new specific question here in Answers, or maybe hop on Splunk's slack channels and ask there!

Happy Splunking
-Rich