Getting Data In

First install problem - Use a deployment server OR a receiving indexer- Which one? Why?

Explorer

Hi all,

I am trialling Splunk and installing it for the first time. I have installed the main Splunk server without problems and found the documentation and videos both useful and very applicable.

However I have been struggling for several hours trying to get the universal forwarded installed and working. And I cannot find any "useful" documentation. I may well have missed something obvious, in fact I hope that is the case.

First time I installed the universal forwarder I configured it to use the deployment server and gave server address and checked the standard 8089 port was listening on the server and that the client can reach it (they are on the same subnet). But netstat showed no comms between the two. So I uninstalled the universal forwarder and reinstalled to point to the server at a listener port. I then realised that this port is not listening on the server - but I though the docs suggested that would be a default listening port at 9997?

So now I am about to set up a port on 9997 on the server. But I am asking myself, why am I doing this? What is the advantage / disadvantage of using a listener or a deployment server port to speak to the client's universal forwarder? It's not clear and I'm getting a bit frustrated with Splunk to be honest.

I am running the server on Ubuntu server 12.04 and the Universal forwarder is on a client running Windows server 2008 R2 core. We have a mixed server environment. I understand that I may need to install the windows app on the server to allow correct indexing?

So if someone could explain to me why should I use deployment server versus receiving indexer? And if I definitely need to install the windows app before I receive the data? Or call me a fool for missing some step by step and/or video demo of someone setting up the universal forwarder?

Any help would be much appreciated.
Thanks,
Jamie

Explorer

Thanks everyone. I have followed the example in the link above and changed the serverclass.conf and output.conf and input.conf where necessary. I now have a basic setup working with one server acting as combined deployment server and indexer. I am collecting windows event logs and a jboss access log. I am doing this as a proof of concept and have more logs and servers to include so I know I will have more questions, especially on congestion and throughput because we can top 1GB of logs an hour per webapp server (multiple by many webapp servers), but I will open these as separate questions.

There is a wealth of documentation on splunk-base which is great except I think for the new starter it is almost overwhelming. I used one of the videos to see a deployment server install which made for an easy to follow tutorial. However there was then no follow up video for adding adding remote data (only local data) unless I missed it somehow. So I do think that although the docs are thorough, there would be benefit to having a single track of different example tutorials that you can just follow along with if your are just starting Splunk.

The data I now see on my indexer is impressive, although again almost overwhelming. I need to learn how to "index" properly perhaps, it has done it by default so far. Also need to learn how to find the most valuable bits, get the dashboards and alerts up. Anyway, so long as it can take the throughput we need to throw at it, it will certainly be a very powerful tool.

Thanks again,
Jamie

0 Karma

New Member

Hey Jamie,

Can you please share the video link for the deployment server installation . It would be really great .

Thanks,
Tushar

0 Karma

Splunk Employee
Splunk Employee

As Steve G. stated, there's no GUI for the Universal Forwarder. There is a GUI for the deployment server... sort of, but only a few of the features are supported in the GUI. If you're interested in using the deployment server, I'd recommend editing the files directly. The link you posted: http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Extendedexampledeployseveralstandardforwar...
is perfect. It shows you how to set up the deploymentclient.conf, the serverclass.conf, and the apps to deploy.

0 Karma

Splunk Employee
Splunk Employee

The universal forwarder doesn't have a UI, alas, so you need to use the CLI (or directly edit the conf files).

Regarding your main issue about the deployment server. The deployment server is a full Splunk instance that has been enabled for sending configuration updates to other Splunk instances, including forwarders. The connection between forwarder and deployment server does this: it allows the forwarder to receive updates from the deployment server. Deployment servers are most useful when you are trying to manage many forwarders.

However, independent of whether you use a deployment server for distributing updates, you also need to set up a connection between the forwarder and the receiving Splunk indexer. The forwarder uses that connection to send data to the indexer. (You can do this by means of the deployment server, but that would only make sense if dealing with multiple forwarders.) For info on how to set up a connection between one forwarder and one receiving indexer, start here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Setupforwardingandreceiving#Set_up_forward...

Please note that you need to first enable a receiving port on the indexer, and then you point the forwarder to that port.

Explorer

I've just seen this...

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Extendedexampledeployseveralstandardforwar...

...it seems as close as I can find to a simple end-to-end walk through example for new people like me. I'd hoped to have seen something similar that would be all gui based but looks like file editing is a must? Personally that's ok, I do a lot of file editing with Nagios but it's a harder sell for the rest of my team.

0 Karma

Splunk Employee
Splunk Employee

jamiemfuse,

And I cannot find any "useful"
documentation.

I'd recommend this page: http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Deploymentoverview

First time I installed the universal
forwarder I configured it to use the
deployment server and gave server
address and checked the standard 8089
port was listening on the server and
that the client can reach it (they are
on the same subnet). But netstat
showed no comms between the two.

The communication between a deployment client and a deployment server is not constant. By default, the deployment client will "phone home" to the deployment server once every 60 seconds to check for new content.

It sounds like you may be confusing a deployment client/deployment server relationship and a forwarder/indexer relationship. They are two different things. A forwarder sends data to an indexer. A deployment client checks in with a deployment server and downloads new apps from it. Essentially a deployment server is a Splunk configuration management tool. In your case, since you're just trying out the software, you're probably not going to need to worry about a deployment server, but you can check out this link for the info on what a deployment server does: http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Aboutdeploymentserver

I then realised
that this port is not listening on the
server - but I though the docs
suggested that would be a default
listening port at 9997?

Check out this link on setting up a receiver:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Enableareceiver

I understand that I may need to
install the windows app on the server
to allow correct indexing?

This is incorrect, but it may be just in the way you're describing it. What are you looking to monitor on the Windows server? If, for example, you just want to monitor Windows event logs, then you just need to set up inputs on the forwarder that define those logs. Also, when you install a Universal forwarder on a Windows server, part of the setup process involves defining some inputs. You can specify Windows Event logs or other logs or directories at the time of setup.

Ultimately it looks like your confusion stems mostly from what a deployment server does vs. what an indexer does. The links I provided should get you on the right track.

Explorer

I've just seen this...

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Extendedexampledeployseveralstandardforwar...

...it seems as close as I can find to a simple end-to-end walk through example for new people like me. I'd hoped to have seen something similar that would be all gui based but looks like file editing is a must? Personally that's ok, I do a lot of file editing with Nagios but it's a harder sell for the rest of my team.

0 Karma

Explorer

I now want to create a really simple config on the deployment server that it can pass to the client when he calls home. I presume the config should contain the indexer address and port and the log name/type to be pushed up to the indexer. It looks like I need to define a "server class" and then a "client config"? Is that right? Could you shed some light on where and how to do that, or a doc that gives a nice example?

Thanks,
Jamie

0 Karma

Explorer

I've read through more documentations coming from the links you gave but a couple of hours later and again I'm struggling. (And by the way I'm about 8 years in IT so which is why I'm so annoyed with myself that I can't get this to work!) I'm just finding that there is almost too much info in the docs, I'm looking for some simple blog-style "I set this up" path of info but not finding it. So you could please give me a few more pointers that would be great.

0 Karma

Explorer

tcp 0 0 10.64.73.210:8089 10.64.73.16:50080 ESTABLISHED

18:50:04

tcp 0 0 10.64.73.210:8089 10.64.73.16:50117 ESTABLISHED

18:57:30

tcp 0 0 10.64.73.210:8089 10.64.73.16:50123 ESTABLISHED

18:58:42

tcp 0 0 10.64.73.210:8089 10.64.73.16:50129 ESTABLISHED

18:59:55

0 Karma

Explorer

Thank you for this. You have definitely moved me forward a bit. I get the difference between the deployment server and the index server now. Ideally I want my proof-of-concept server to do both jobs.

I reinstalled the universal forwarder again with only the deployment server details and yes you were right it is calling home every once in a while, here is the comms on the deployment server:

0 Karma