Textfile managed via Splunkweb UI

[sdwilkerson]

Can you create a Splunk View that would allow users to manage a text file (e.g. a CSV used for a lookuptable)?

The Use Case here is a Watchlist that triggers on specific threats but managed manually by some of the slunk users via the Splunk Web UI. Mgmt through the UI is preferred over scping a CSV to the system every day.

Even if the Splunk web syntax doesn't support this, I know it isn't that difficult in python. If this was the only route, how much effort might it be to integrate?

Sean

[Lowell]

This is just a thought off the top of my head, and it may be awful or impractical but I'm pretty sure it's technically feasible. (end of disclaimer)

Instead of trying to create something new, or doing a bunch of UI development, what about a completely different approach. What if you used an online service, like google docs? You could could upload your existing lookup (.csv) file, and easily make changes from a web browser. You may even be able to embedded the google docs interface within a splunk UI page (not sure about that), but certainly you could simply stick a simple link to your doc and have it open in a other window. Goggle docs let you could control who is allowed view vs edit access, and can even setup notification emails when changes are made. I think that google lets you use build a static URL to a CSV version of your document so that you could a small script (or possibly something as simple as wget) to download your doc in CSV format, and then save it to the lookups folder. And if you can't do this directly, I know that google provides lots of APIs for this kind of thing, so it's certainly possible to do with some short python scripts.

You could use either a scheduled job, or some kind of UI based trigger that would pull down the most recent version of your lookup file from google docs.

Of course this introduces data sensitivity questions and online access issues from your splunk instance which you will need to consider. But on the other hand, you could whip up a solution in a few hours vs spending lots of time trying to build something from scratch (or even leveraging other peoples tools.) You also don't have to worry too much about stack upgrades, since I'm doubting that google will be making many incompatible API changes.

---

Some helpful URLs:

• http://gdatacopier.googlecode.com/ - This provides a very simple gcp.py utility that will let you easily download your google spreadsheets in CSV format (it's also smart enough to only export the file if the online version was updated.)
• http://code.google.com/apis/documents/developers_guide_python.html

[sdwilkerson]

Interesting idea. Unfortunately, in this case, there is a sensitivity issue so an external service won't work.
This is a solution they want for the future, but not right away, so I have some time to think about other possibilites

[gkanapathy]

If you want a UI to edit rows and columns, that is a lot of work (as UI work tends to be, though some development environments provide you with a lot of tools to make it easy). It's not clear to me what you mean by "managing" a text file. It's also not clear to me what would constitute "integration". However, anything you do basically involves building a complete program to edit a text file.

You can use the outputlookup command to overwrite a Splunk lookup file, usually run periodically with a saved search and using data from a Splunk search result, and often merging it into the existing lookup by using inputlookup in a subsearch. That's the extent of it in Splunk.

I would recommend against trying to build anything complex like this into the Splunk web.

[Lowell]

That is one nice thing about CSV files. You really do have lots of options. Although, some times having lots of options means there you end up spending more time talking people out of the bad ones. 😉 Good luck. (If you end up with something you think others would benefit from, please post you own answer here.)

[sdwilkerson]

I understand. The backup plan is to build something in-house in cgi-bin/edit.pl.
I have spent the last couple of weeks trying to convince them to do more in Splunk and less with external scripts and wanted to follow that theme.
Thanks again for quick responses.

[gkanapathy]

It may sound easy, but it's very difficult (and yet more difficult to do securely). I would look for some other app server, plugin, web server, ftp server+client, or anything else that is already built that would give you editing abilities to files on your server. (I doubt you'll find anything for CherryPy, and even if you did, you will have a hard and non-upgradeable time putting it into SplunkWeb.) I'd install and configure that before I tried to make anything like that in SplunkWeb. Personally, I would just use FTP and have people edit via the FTP client built into basically every OS.

[sdwilkerson]

Gerald,
Thanks for your response, this helps.

Refined Use Case: Imagine some outside entity sends you regular updates as to suspisous activity that you want Splunk to alert on positive hits. You need a way to easily update the lookuptable NOT based on info already in Splunk. A webui would be easy for group administration.

Could something in splunkd UI allow this? Doesn't need fancyness, just ability to edit a file.csv thru UI.

Sean