All Apps and Add-ons

Splunk Add-on for NGINX: How can we forward the server data to the add-on installed on the Splunk server?

g4s
Engager

Hi,

I'm a newbie - I was facing issues while forwarding custom nginx data to the Splunk Add-on for NGINX installed on my Splunk server.

-- I went on reading a lot of things, Some points which i think can be a viable solution, But I wanted to be sure before doing anything or Better if anyone can suggest me a simple solution.

Points -

** Install the add-on on the box to be monitored by using 'Heavy Forwarder' / Search Heads or by installing a separate Splunk web version ??

Confusion -

  • How to sync the data between the add-on on the box with the add-on on the Splunk server?
  • What should be the " [monitor:///] "
1 Solution

jonmargulies
Path Finder

You're going to do this in two parts:

1) Getting the logs in: In Splunk_TA_nginx/local, you're going to create an inputs.conf file. Here are instructions for the contents of that file: http://docs.splunk.com/Documentation/AddOns/released/NGINX/Configureinputsv2monitor. Copy the monitor stanzas from that link exactly, but replace with the actual path to the nginx logs (if you want a different index than main, these stanzas would be the place to set that as well). For example, if your nginx access logs are in /var/log/nginx/access.log, that monitor stanza should begin [monitor:///var/log/nginx/access.log] (note the number of slashes; getting that number right is important). In order to get Splunk to collect nginx logs, you need to deploy the TA (especially that local/inputs.conf you just created!) to forwarders on every host that generates nginx logs.

2) Parsing the logs: The rest of the TA is all about telling the search head how to parse nginx logs and break them into fields. For that to work, all you have to do is install the TA on your search head(s).

View solution in original post

jonmargulies
Path Finder

You're going to do this in two parts:

1) Getting the logs in: In Splunk_TA_nginx/local, you're going to create an inputs.conf file. Here are instructions for the contents of that file: http://docs.splunk.com/Documentation/AddOns/released/NGINX/Configureinputsv2monitor. Copy the monitor stanzas from that link exactly, but replace with the actual path to the nginx logs (if you want a different index than main, these stanzas would be the place to set that as well). For example, if your nginx access logs are in /var/log/nginx/access.log, that monitor stanza should begin [monitor:///var/log/nginx/access.log] (note the number of slashes; getting that number right is important). In order to get Splunk to collect nginx logs, you need to deploy the TA (especially that local/inputs.conf you just created!) to forwarders on every host that generates nginx logs.

2) Parsing the logs: The rest of the TA is all about telling the search head how to parse nginx logs and break them into fields. For that to work, all you have to do is install the TA on your search head(s).

View solution in original post

g4s
Engager

Hi,

I created the search head, But somehow unable to forward the data to my indexer.

Some help is required --

-How should we make sure that 1 instance can only remain as an indexer. What is required to be changed in configuration ?

-Do we require outputs.conf file on search-head to forward the data to the indexer ?

I was able to create a search head & took good amount of time in setting the TA across the forwarder also. But nothing came up on the search-head. I'm going to do more troubleshooting on this.

0 Karma

jonmargulies
Path Finder

Do you have an outputs.conf on your forwarder that tells it to send data to your indexer? If not, here's the basic documentation on doing that: http://docs.splunk.com/Documentation/Forwarder/latest/Forwarder/Configureforwardingwithoutputs.conf

Here's the flow: The forwarder is installed on the host that generates nginx logs. It has an inputs.conf (defined as I described above) that tells it which log files to watch, and an outputs.conf that tells it what indexer to send all its data to.

The search head will peer to the indexer (assuming you have distributed search configured) and allow you to query the indexer for your data.

Now, to answer your specific questions:

How should we make sure that 1 instance can only remain as an indexer. What is required to be changed in configuration ?

You configure a general Splunk Enterprise instance to be an indexer by giving it an inputs.conf and telling it to listen for data. The basic instructions are here: https://docs.splunk.com/Documentation/Splunk/6.5.3/Forwarding/Enableareceiver

By default, Splunk Enterprise has an index called main and some internal indexes, so all of the data from your forwarders will go into that main index. If you want to use custom index names, instructions are here: http://docs.splunk.com/Documentation/Splunk/6.5.3/Indexer/Setupmultipleindexes

Do we require outputs.conf file on search-head to forward the data to the indexer ?

Outputs.conf is required on any host that should be forwarding logs to another host (typically that receiving host will be an indexer). So in a typical configuration, you'll have an outputs.conf configured on all of your forwarders, as well as on your search heads (so they can send their own logs to indexers to be indexed).

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!