All Apps and Add-ons

Managing DB Connect and Heavy Forwarder in a distributed environment for reading logs

afx
Contributor

Hi,

we have an application that insists on writing relevant log information to an MS SQL DB instead of decent files.
This of course means I need DB Connect to read the logs. I am not interested in direct lookup from the search head.

So how do I set up the Heavy Forwarders in our environment?
I have two Indexers, a search head and a deployment server.
I definitely do not want additional hardware or VMs.

My plan is to install the HF and DB Connect on the application server just like I would install a UF.
And then have it connect to the applications DB System via JDBC. The Heavy forwarder should of course also forward the local logs from the application server and not just the log extracted from the DB. The application server has some extra capacity to handle the load.
I would prefer to manage the HF just like a UF via deployed configurations without any GUI active on the HF and no local configuration apart from the initial install of HF and DB connect and the setup of the deployment client configuration.

Is this possible or do I absolutely need the GUI on the HF to set up the DB Connect or other things?
Can a configuration for the DB connect be pushed via some other deployment app (as the DB Connect itself is not deployable)?

thx
afx

0 Karma

afx
Contributor

Ok, tried it and failed.
Key issue for not being able to use the deployment server is the DB password. The pw data sent by the deployment server needs to be hashed with the seed from the UF. Yes, that could be hacked but is not very practical.
And it seems that when using rising column mode some file gets initialized via the GUI which I have not been able to create manually.
And currently the GUI is broken thanks to Python 3 issues ;-(
cheers
afx

0 Karma

isoutamo
SplunkTrust
SplunkTrust
0 Karma

afx
Contributor

Thanks, that worked.
This leaves only the rising column tracker files. Copy and paste of the first line from a system where I used the GUI seems to be the way to go.
thx
afx

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi afx

Your plan is ok. You could do is just as described above.

Anyhow personally I propose to use GUI for configuring inputs and connections. It also contains Health monitoring part. Of course You could/should install that also to your SH for Monitoring and see current status of DBConnection. There haven't been any reason to use those lookups or other additional features in SH. Don't use those inputs on SH side, those must do on HF side on distributed environment.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...