Need deployment server to assign hostnames dynamically via inputs.conf

Jaci — Sat, 15 May 2010 05:04:25 GMT

I have a deployment server app with a single inputs.conf file.

[tcp://localhost:9997]
sourcetype = tcp-raw
index = pp-dev

A windows .NET application writes to this port with XML data. Splunk is indexing the data correctly, although the host is showing up as 127.0.0.1. I have this app deployed on about 50 hosts via the deployment server. Is there a way for the Splunk to show the hostname vs. 127.0.0.1 on the indexer?

I am looking for a way to dynamically assign the name. If I have to hardcode the name, then this defeats the benefits of the deployment server.

Re: Need deployment server to assign hostnames dynamically via inputs.conf

Lowell — Sat, 15 May 2010 05:36:44 GMT

Great question. I doubt it's possible. The deployment process seem to be pretty limited, IMHO.

WARNING: UGLY UGLY HACK...

I guess you could do a hack with an input script...

Make sure your existing inputs.conf is in the default directory. Then write a simple python script to check for the existence of the local/inputs.conf in your app. If it does not already exist, then your app should create it with the following template:

[tcp:://localhost:9997]
host = <host to be dynamically filled in by script>

After the local/inputs.conf file is written, it should issue a splunk restart command so that the local entry takes effect. When splunk startup up again, the script will run again, but this time since local/inputs.conf exists, it will not make any changes or restart splunkd. (You really don't want a recursive restart loop. That would be bad, especially on 50 machines).

You would probably want to schedule this script to run like once every 31536000 seconds (every year); so that it will only effectively run once a splunkd restart.

When you redeploy your app, obviously your local/input.conf will be wiped out. This shouldn't be a problem though because splunkd will have to restart once (oh yeah, make sure you have restartSplunkd=true in your deployment config). So when splunkd restarts with the newly deployed app, since local/inputs.conf will be missing again, the file will be written and splunkd will be restarted again, but this time with your proper host entry...

Wow! That's ugly. not as bad as it could be, definitely doable, but certainly not pretty.

Do you have any python experience? I could probably whip up a prototype if you want. I know there have been times where I've wanted a feature like this before.

UPDATE... I went ahead and wrote a quick python script that should get the job done (I haven't actually tested it, so there could be issues.)

""" hack_local_inputs.py:  Simple hack to write out a local/inputs.conf file
for this app with a hardcoded host value.
"""

import os
import socket
from subprocess import call

APP_NAME = "my_app_name"
SPLUNK_HOME = os.evniron["SPLUNK_HOME"]
HOSTNAME = socket.gethostname()

conf_file = os.path.join(SPLUNK_HOME, "etc", "apps", APP_NAME, "local", "inputs.conf")

if os.path.exists(conf_file):
    # This is for debugging this script
    print 'Nothing to do here, conf file already exists...  conf="%s"' % conf_file
else:
    stream = open(conf_file, "w")
    stream.write("[tcp:://localhost:9997]\n")
    stream.write("host = %s\n" % HOSTNAME)
    stream.write("disabled = 0\n")
    stream.close()
    # This is for debugging this script
    print 'Finished Hardcoding host hack!  splunk_app=%s new_host=%s conf="%s"' \
            % (APP_NAME, HOSTNAME, conf_file)
    splunk_exe = os.path.join(SPLUNK_HOME, "bin", "splunk")
    call([splunk_exe, "restart", "splunkd"])

Then, in your default/inputs.conf add something like this:

[script::$SPLUNK_HOME/etc/apps/my_app_name/bin/hack_local_inputs.py]
sourcetype = hack_local_inputs
interval = 31536000

Re: Need deployment server to assign hostnames dynamically via inputs.conf

Jaci — Tue, 25 May 2010 03:58:07 GMT

Thank you for the answer and the python script.

topic Re: Need deployment server to assign hostnames dynamically via inputs.conf in Getting Data In

Need deployment server to assign hostnames dynamically via inputs.conf

Re: Need deployment server to assign hostnames dynamically via inputs.conf

Re: Need deployment server to assign hostnames dynamically via inputs.conf