About sideview

sideview · ‎02-19-2016

Actually you can get what you need by using a ValueSetter module with a little conditional logic in it. Say you have a key called "foo" upstream, and you want the Gate to open if foo is null, and be closed when foo has a value. <module name="ValueSetter"> <param name="name">isFooNull</param> <param name="if.$foo$=*"></param> <param name="default">yes!</param> downstream from this ValueSetter you can use $isFooNull$ and it'll only have the "yes!" value when $foo$ is null. This should do it. ValueSetter is exactly for tweaky little tasks like this. In the docs it has a number of different sections going through it's various features. If you've missed the conditional stuff, some other things in there might be useful to you as well. Also if you'd prefer to get nitty gritty working examples, there's a hidden view that ships in the app called "testcases_for_value_setter_conditionals" that might be useful for cribbing working config from.

sideview · ‎01-26-2016

UPDATED. Yes it is possible. use the servicesNS endpoint for conf-inputs and pass "nobody" as the user. If you're used to the old /services/data/inputs/* endpoints, and you've had services!=servicesNS confusion in the past, beware that you don't want to use "/services/configs/conf-inputs" only use servicesNS/nobody/search/configs/conf-inputs In brief, and thanks to Support for this example: > curl -u admin:changeme -k https://127.0.0.1:8089/servicesNS/nobody/search/configs/conf-inputs -d 'name=batch:///home/work/305482/batch_fodder' -d 'move_policy=sinkhole' > cat ./splunk/etc/apps/search/local/inputs.conf [batch:///home/work/305482/batch_fodder] move_policy = sinkhole Less Interesting Other Details. You can't use the actual Data Inputs endpoints. If you use the actual input endpoints and not just /configs/conf-inputs http://docs.splunk.com/Documentation/Splunk/6.2.7/RESTREF/RESTinput , it's a bit weird. Batch/sinkhole inputs will get listed out as though they were "monitor" inputs. This seems to be an unrelated bug (call it Bug #1). If the developer notices that some of the inputs are coming back with "move_policy" set to "sinkhole" (not a legal key for a real monitor input), they could deduce that this must not be an actual monitor input but rather a batch input hiding away. Anyway this inspired a lot of hope that there was some hidden secret way you could POST a new input to /services/data/inputs/monitor and end up with a batch input created but no. /services/configs/conf-inputs This "works" but makes sure you use servicesNS and use "nobody" as the user. With some other combinations it will physically create a stanza in an inputs.conf file but it will create the stanza in etc/users/Username/Appname/local/inputs.conf, and batch inputs cannot be specified there, splunkd ignores them completely, does not run them, will never list them. Very confusing.

sideview · ‎01-18-2016

That's basically the solution I have now, but since this is an app it gets set up in the field. There when they skip the step or they misread the underlying problem as being just about disk size, it becomes a problem. There can very quickly be tens of thousands, hundreds of thousands, millions of these files in these directories and obviously Splunk doesn't react very well when it's constantly checking huge numbers of files for appended changes. Indexing falls behind, IO performance on the box becomes awful. It's a perfect case for a sinkhole input.

sideview · ‎01-12-2016

Yep. I thought of that. Using the entity class and then later using raw splunk.rest code, it doesn't matter. Submitting to the correct non-namespaced endpoint (/services/configs/conf-inputs), where it should definitely be putting the stanza into the app directory (or system directory), splunkd always puts the input stanza over in etc/users/admin/search/local/inputs.conf. As I mentioned it's DOA and invisible because you can't have batch inputs in user conf and Splunk steadfastly ignores it. I tried the Python SDK but it looks like (and I could be wrong), its implementation to wrap the conf-* endpoint seems to not match the actual behavior of that endpoint.

sideview · ‎01-11-2016

Actually I've had no success working around the second problem. Various combinations of setting and not-setting owner and namespace, or setting owner to "nobody", crossed with setting 'eai:acl' to "app", "global", "system" etc, all such permutations result in the data input conf being created at the user-level where Splunk ignores it, rather than at the app or system level where I need it. Fixing the sharing by modifying "eai:acl" on a subsequent rest call with Entity.py isn't possible because since splunkd ignores inputs.conf stanzas there, it doesn't see it. Therefore you get a 404 on the subsequent get call. The only recourse seems to be doing filesystem operations directly to write the raw config and prompt for a server restart after.

sideview · ‎01-11-2016

Update - this does work, with the following drawbacks. 1 - it requires that the user have the admin_all_objects capability, which in larger deployments is severely restricted and which even admin level users may not have. 2 - When created, (at least with Entity.py) it is created as a namespaced object, and Splunkd puts the inputs.conf file in $SPLUNK_HOME/etc/users///inputs.conf Which isn't particularly useful, because Splunkd ignores these stanzas here entirely. The only way they can be seen by Splunk is if they're created at the app level. I think the second problem can be worked around by crufting in a few subsequent rest calls to modify the eai:acl key. =/ But since the first problem is kind of a deal breaker I haven't gotten around to this yet. Does anyone have another answer, or are there examples of using the Splunk Python SDK from within a custom python controller in a Splunk app? Maybe I'd have better luck using the (at least newer) Python SDK.

sideview · ‎12-21-2015

It should be fine. However 3.2.7 is a year and a half old and I recommend upgrading to the most recent Sideview Utils (3.3.8) Release notes are here - http://sideviewapps.com/apps/sideview-utils/release-notes/ and you can add yourself to the release notes mailing list here - http://sideviewapps.com/apps/sideview-utils/mailing-list/ Your email will never end up anywhere else and you wont get any emails out of it besides the dry little release-notes email for every maintenance release.

sideview · ‎12-18-2015

I maintain an app with a data input wizard, under the hood of which is a custom controller that can list and create normal "monitor" data inputs very reliably. I now need to expand this to listing and creating "batch" data inputs aka "sinkhole" inputs, and from what I'm seeing maybe this isn't even possible. The REST API Path for normal monitor inputs is /data/inputs/monitor and in inputs.conf these look like: [monitor://D:\some\path\*] index = foo sourcetype = bar I now need to be able to also list and create "batch" data inputs. In inputs.conf these end up in the form: [batch://D:\some\path\*] move_policy = sinkhole index = foo sourcetype = bar I cannot use a oneshot input. In this particular use case, files are ftp'ed to the given directory every minute or so in real time and I want to delete them as they are indexed. Hence batch is perfect. 1) Listing batch inputs: There seems to be no separate endpoint for these, but they do get listed (oddly) in the output results for /data/inputs/monitor. So, the only way I've found to list them, is to request /data/inputs/monitor and then look for the "move_policy" key being set to sinkhole. Q1: Is there a better way? 2) Creating batch inputs: I have not found any way to do this. There is no /data/inputs/batch or /data/inputs/sinkhole endpoint and thus no /data/inputs/batch/_new to POST to. Following the weirdness of #1 above, if I use the entity code to create a "_new" monitor input, and I sneak in a "move_policy" key set to "sinkhole", that doesn't work either - it just complains Argument "move_policy" is not supported by this handler. Q2: Am I missing something? Is there a way to actually do what I need? Eager for any help or any advice. If I have to conclude that such a simple data input administration task is impossible, I'll be sad and I'll have to just resort to doing filesystem operations to write the conf files directly through popen, and then prompt the user to restart the server afterwards.

sideview · ‎12-01-2015

At present, not without writing a CustomBehavior and putting it into application.js. However this is a very reasonable request and I've actually run into this myself and have a couple customBehavior's out there doing similar things. I have gone ahead and added the feature, testcases and docs and it will release in the next version of Sideview Utils to go out, probably in a few weeks and currently slated to be 3.3.9 although the exact version number is subject to change. Add yourself to the release notes mailing list if you aren't already. ( http://sideviewapps.com/apps/sideview-utils/mailing-list/ ) If you would like to test out the feature in advance of the next version, email me at nick [at] sideviewapps.com and I can send you a build of the app.

sideview · ‎11-13-2015

It looks to me like there are no spaces between the asterisk characters? ie it looks like "1*****" when it should be "1 * * * * *". This wouldn't explain why the client is not displaying any helpful error to that effect so perhaps it's not the only problem.

sideview · ‎11-12-2015

I'm sorry but this question doesn't make any sense so I think you're just asking it in a confusing way. "the average of events with Success, the average of Errors" makes very little sense. Do you want to end up with a single overall average success rate like 37%? If so then index=main sourcetype=performance Status=* | eval foo=1 | chart count over foo by Status | eval ratio=100* (S/E) | eval ratio=ratio + "%" will do the trick. But this wouldn't incorporate your other requirement, "compare this average to last event so I can get the average difference to the last event" which still doesn't make sense.

sideview · ‎11-12-2015

Are these individual events single-line events with only one value for Status each, that then are written out in big groups of events, where each group for you constitutes some "event" in the real world? Or are these events large multiline events with multiple values of Status in each?

sideview · ‎11-12-2015

Heads up you can also always use the search command again to filter down to just the row(s) you need, even aftert they've been grouped and transformed by other commands. eg here | search Rating=="Total" would do it too.

sideview · ‎11-11-2015

You can use streamstats but you first need to reverse the sort order. | sort + _time| streamstats last(client_type) AS client_type BY account_id| stats sum(amount) By client_type

sideview · ‎11-11-2015

Hmm. Well I would verify that there is in fact more than one EventCode for that combination of index source and host, by running this index=xxxx sourcetype=yyyy host=zzzz | stats count by EventCode If there's more than one EventCode value in the set then the streamstats+search combination in my answer should be working. Here's another related example you can play with yourself that uses Splunk's index=_internal data. index=_internal | head 10000 | streamstats global=f count as i by sourcetype | search i<4 | sort sourcetype In the most recent 10,000 events it returns the 3 most recent events for every sourcetype.

sideview · ‎11-11-2015

this will work. index=xxxx sourcetype=yyyy host=zzzz |streamstats count as i by EventCode | search i<4 | fields host,EventCode,EventCodeDescription streamstats goes through your rows and with each distinct EventCode treated separately, it puts a field called "i" on each row whose value is the number of rows that have had that EventCode value on them. ie, for EventCode=12 , the first such row gets i=1, second gets i=2 etc. Then we just let it go through the whole set, and then use search i<4 to throw away everything that wasn't the first three representative events for the given code. http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Streamstats

sideview · ‎11-06-2015

hehe. Sure no problem. He and I were writing our answers at the same time. Unaccepting answers and accepting others happens all the time but in this case they are both right so it matters little. Cheers.

sideview · ‎11-06-2015

Well you have the time of the event as _time field, and you can use now() in eval expressions, so you can make a field, let's call it secondsAgo, like so: | eval secondsAgo=now() - _time It is of course just a number of seconds. IF you were to do | convert ctime(secondsAgo) , that would be weird because you're asking Splunk to tell you what time it would be if this number of seconds were defined as "the number of seconds since 1/1/1970 in GMT", which.... is generally a random time in 1970. What you probably want to do after getting secondsAgo as an integer, is convert it to an "HH:MM:SS" duration string, like so: | eval secondsAgo=now() - _time | eval durationStr=tostring(secondsAgo,"duration") or if you prefer it in one eval expression, | eval secondsAgoStr=tostring(now() - _time, "duration")

sideview · ‎11-05-2015

You have a couple options. If you have an app called myapp, then 1) The newer way (6.0+ or possibly back to 5.0), putting it in $SPLUNK_HOME/etc/apps/myapp/static will give you a URL of: http://searchhead:8000/en-US/splunkd/__raw/servicesNS/admin/myapp/static/health_check_api.html 2) Or the older way, you can put it in $SPLUNK_HOME/etc/apps/myapp/appserver/static and this will give you a URL of: http://searchhead:8000/en-US/static/app/sideview_utils/health_check_api.html It's possible that someday Splunk might remove the legacy appserver/static path so maybe #1 is better, but this is just speculation. And as you might guess from #1, there is an equivalent URL on Splunkd's port to the same file https://searchhead:8089/servicesNS/admin/cisco_cdr/static/appIcon.png (note the port is the 8089 port here and not the 8000 port) but of course you would need to pass HTTP auth on that request and that probably isn't feasible.

sideview · ‎11-02-2015

I looked into this and you would have to modify the code in update_lookup.js to accomplish this. The key part, ie to actually prepopulate the given fields (I assume btw that you mean to prepopulate them whenever a user adds a new row) -- you'd have to add a customBehavior to the Multiplexed HTML module that represents the fields of the "Add New" form, and within that customBehavior implement the HTML module's optional "onHTMLRendered" method. For the docs about customBehavior go to Tools > Other Tools and scroll to the bottom panel. And Sideview Utils itself has quite a lot of customBehaviors throughout that you can read as examples.

sideview · ‎10-28-2015

If your string formatted time is of the form "2015-10-28 08:52:41", then | eval epochTime=strptime(timeStr, "%Y-%m-%d %H:%M:%S") If you need to convert multiple formats, you'll need multiple eval clauses. docs for all of the functions that eval can use: http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/CommonEvalFunctions You'll also probably find the quick reference guide handy. Note the last page which has all the common timeformat %X values. https://www.splunk.com/web_assets/pdfs/secure/Splunk_Quick_Reference_Guide.pdf And at pretty much any splunk event they often hand the reference cards out on 8.5"x11" cardstock.

sideview · ‎10-27-2015

Assuming what 6.X does is similar to what 4.X and 5.X did, the UI actually gets from splunkd a json representation of the current timezone's entire olson table, containing all dst offset changes. This gives it the ability to correctly translate any epochtime value to a string-formatted time. However... if the operating system's idea of where timezone offset boundaries are is weird or wrong (or in this case insanely wrong), the potential for havoc is somewhat limitless. Sounds like you found a workaround to either change the timezone or tweak dst handling on the host to make the bug go away.

sideview · ‎10-27-2015

Likewise, thanks for the cleanup.

sideview · ‎10-27-2015

Well, to be honest, it would be extremely difficult because this is not the way you want to do this, at all. Backing up, there is a DateTime module in Sideview Utils (if you don't see it then you're on the ancient LGPL version and you should update to the free 3.X version here: http://sideviewapps.com/apps/sideview-utils/) The DateTime module is configurable to allow just dates, or dates plus times to be selected. It can also be used in pairs thus to pick both start time and end time. If DateTime isn't your cup of tea for some reason the way you would extend a sideview XML view with custom Javascript is through a customBehavior module. This allows for a much cleaner implementation, with JS living in application.js. You can find a docs page talking about customBehaviors if you go to Tools > Other Tools, then scroll down to the bottom and click the link you see there. CustomBehaviors can be very simple or very complex but there are a lot of examples hidden away inside Sideview Utils. One more note - in general you want to avoid the HTML module for custom stuff like this, because when the HTML module receives a push from upstream, all the HTML inside will be thrown away and rewritten. This almost always has crippling effects on custom code like this. In this case your JS would execute again, and even when the JS is off in application.js, recreating the HTML will usually reset the state as well as sever the relationship between the HTML in the page and the JS that's trying to modify it.

sideview · ‎10-26-2015

Wow. 1) Can I ask what timezone your Splunk Server is in? 2) Can you also double check what the host thinks the current time is? My only idea is that the rather-complex timezone and daylight-saving-time code is being thrown off WILDLY, possibly contributing to this would be if you were in a rather unusual timezone? 3) Can you also export these results as CSV? I'm curious to see the raw integers of the timestamp values here. If they are actually off as well then, er, even more wow.

Posts	3349
Solutions	892
Karma Given	2122
Karma Received	2938
Member Since	‎11-07-2009

Online Status	Offline
Date Last Visited	a week ago

How to deal with alerts that can't be enabled in t...

Why is my python script suddenly writing "Authoriz...

Looking to rewrite a join across several indexes w...

To what degree does chaining (or nesting) automati...

Why do I get "error parsing XSL Stylesheet" when I...

Finding and reporting on callers getting bounced a...

Callmanager CDR - whether we can do reporting on t...

Callmanager CDR - reporting and alerting around sp...

Strange relationship between fieldformat command a...

calculating the total time an extension has spent ...

Re: Sideview Gate - Feature Suggestion - nullKeys

Re: Is it possible to create a batch data input vi...

Re: Is it possible to create a batch data input vi...

Re: Is it possible to create a batch data input vi...

Re: Is it possible to create a batch data input vi...

Re: Is it possible to create a batch data input vi...

Re: Is Sideview Utils 3.2.7 compatible with Splunk...

Is it possible to create a batch data input via th...

Re: Sideview Utils: When all items are selected in...

Re: Why are we not able to schedule a report in th...

Re: How to get the average of two fields and compa...

Re: How to get the average of two fields and compa...

Re: Display Only the Total Row

Re: Is there a design pattern for join searches wi...

Re: How do I write a search to get only the first ...

Re: How do I write a search to get only the first ...

Re: How to find elapsed time between now() and eve...

Re: How to find elapsed time between now() and eve...

Re: Path on server for the url: http(s)://Search_H...

Re: How to use the Sideview Utils updater tool to ...

Re: How to convert a time field with multiple form...

Re: Problems with earliest consultations higher th...

Re: Problems with earliest consultations higher th...

Re: How to use the value of a text box defined in ...

Re: Problems with earliest consultations higher th...

Join the Conversation