Context
The modular input fetches data from a REST endpoint exposed by some other service (similar to reaching out to fitbit to grab your activity data – but this is NOT fitbit). When the data is pulled in I have to make sure only net-new data gets indexed – otherwise there will be duplicate events. To provide the user maximum control, the modular input has a checkpoint-like attribute to keep track of where it left off and only pull in net new data. Therefore, after each pull of data, the modular input should update that checkpoint with a new timestamp of the latest event. That’s why the modular input is attempting to update the attribute’s property.
I had previously used the checkpoint feature for modular inputs but that doesn’t really provide the end user an easy way to manage the checkpoint itself. http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ModInputsCheckpoint
I also considered simply running a search against the input’s defined index/sourcetype/source combo and using the latest event’s time for the same purpose. I deliberately avoided that because I anticipate users would request a way to reindex data in ways I can’t predict.
Solution Attempt
I've created a modular input but I am unable to update one of the input stanza's attributes.
To test this out, I’ve been using splunk’s this command: $SPLUNK_HOME/bin/splunk cmd splunkd print-modinput-config myinput myinput://test | $SPLUNK_HOME/bin/splunk cmd python $SPLUNK_HOME/etc/apps/myapp/bin/myinput.py
To debug, I’ve been going into python CLI ($SPLUNK_HOME/bin/splunk cmd python) manually creating a connection (client.connect) and then trying to iterate through objects until I find what I think looks right.
Let me know if you recommend stronger ways to test/debug. I miss IDEs and seeing the objects to know what properties exist and their values. As far as I understand, there is no good IDE for what I’m trying to do. Lemme know if I’m wrong.
So here’s the issue, I connect to my instance to debug:
service = client.connect(username='admin', password='changeme’)
Then, to validate that I’m connected, I run (is there a better way to validate the connection?):
for item in service.apps:
print item.name
Like the REST API Modular Input, I then use inputs.__getitem__ and I validate with myinput1.content where I see the attribute myattribute has a value of 0 (which is correct).
But then when I run either updates below, I get an HTTP 400 Bad Request.
myinput1.update(**{'myattribute':'500’}),
myinput1.update(myattribute="500”)
Conversely, I have no issues doing the same with an out-of-the-box file monitor input. I can change attributes without issue.
input = service.inputs.__getitem__("normalinput")
input.update(host="test")
Help?
Any ideas? Did I miss a step in my definition to properly expose an endpoint of my input? Like, is there something that needs to occur in the definition of a modular input to allow it to be modified with such a call?
Thanks for any help!
