We have a LWF on Linux that is forwarding to our indexer. We're a little tight on space, but in my experience the LWFs don't use up too much space. On this particular LWF, the metrics.log file is growing at a much higher rate than our AIX-based LWFs. Now we're at risk of filling up the file system. Is there a easy/safe way to clean up these metrics.log files? Is there a reason that file would be growing much faster on Linux than on AIX? Thanks! ... View more

I am writing an app for my team to use. Let's call the app xyz. The app will make use of various inputs, saved searches, etc... I've already deployed a bunch of inputs/"apps" using the Deployment Server. For example, to search for fibre channel adapter errors, I created an app called "fc". It resides in /splunk/etc/apps/fc. There's a Weblogic app in /splunk/etc/apps/weblogic. Etc... Keep in mind I'm only using one index (main). I want the xyz app to make use of the data already indexed by those deployed apps; I don't want my xyz app re-indexing that data specifically. Please correct me if I'm wrong, there's nothing wrong with that approach, right? However, as a best practice, should I consider indexing our "custom" data (fc, weblogc, etc...) into another index? My team is the only one using Splunk presently so protecting the data from others isn't an issue really. Thank you very much. ... View more

I recently made a stab at porting the lsof *nix app to AIX. I realize this is an unsupported configuration, but we AIX users feel left out! Anyways, it wasn't that hard to port. We already had lsof for AIX compiled. I just modified common.sh to fake it into believing it supports AIX, copied the props.conf, and off I went. It runs lsof.sh and indexes the information, but I guess I was expecting more. Maybe I have more work to do on porting it, but for now it seems to just run lsof and captures the output of the command into a single 500 line entry. No special fields or anything like that. Is that the expected behavior of lsof for *nix? Or is there more to it that I am missing? What is the difference between lsof for *nix versus running lsof.sh as your own app? Thanks! ... View more

I am using the Manager to set-up a saved search/alert. Splunk runs a script every so often with an output like this: Active channel: primary channel If "primary" ever changes to "backup", it alerts us via e-mail. "primary" is in a field called "ent_status". In the manager, I created a search like this: sourcetype="echk" ent_channel=backup Using the menus-for-dummies, I told it "if number of events is greater than 0", send us an e-mail. Works great. But now I may be using a third party app to throttle the alerts (see my other question from earlier this morning). I need to re-format my alert to put into the "if custom condition is met" field. I'm having trouble doing this because "ent_channel" isn't an integer; I don't know how to do a compare. How do I translate "if number of events is greater than zero" into a search/alert command? I have the feeling I'm making this harder than it really is. Thank you very much. ... View more