I'm using a Windows Splunk server to collect WMI data. How can I use that to send data to my main Splunk installation, which is Linux? I'm looking for the best practice- what's the easiest way to do this? Install a lightweight forwarder on every Windows server? ... View more

I have a field that should be increasing - though not monotonically increasing. a=1 a=4 a=9 a=13 a=14 a=10 a=101 I want to alert if the number gets smaller. Here's my attempt at this. index="myindex" | stats max(a) as maxnum | where a < maxnum That doesn't find it because the number never decreases below 101. However, even if I change the query, it doesn't find any rows! index="myindex" | stats max(a) as maxnum | where a < (maxnum-100) Obviously, that doesn't do what I want. But it was an interesting diagnosis. I was inspired by this question, but I can't change it to do what I want. Can/should I do this with a subsearch? ... View more

This should be easy. I'm building a query: index=asdf "search string" | rex field=_raw mode=sed "s/.*foo(.*?)bar/\1/" Great. Now it returns a portion of the log that I want to see. I need to "count and group" that now. index=asdf "search string" | rex field=_raw mode=sed "s/.*foo(.*?)bar/\1/" | stats count by _raw Hmm. That just returns raw counts, but not the data. Forcing it causes nothing to return: index=asdf "search string" | rex field=_raw mode=sed "s/.*foo(.*?)bar/\1/" | stats count by _raw as dcount | fields _raw So, given the first query, can someone add "count and group, sort by top counts" to it? Yes, I know this can theoretically be a field extraction, but I don't want that in this case. ... View more

I have a couple of indexers behind a heavy forwarder, which reads from a batch of ports (and a few directories). If both of the indexers go down, incoming data is dropped on the floor. Is it possible to configure my forwarder to queue messages? It's bored, since it is just doing some basic parsing and throwing data towards the indexers. I can't be the first person to consider doing this. ... View more

I want to use the coldToFrozenScript to simply move from cold to my thawed path. Great, my shell script simply contains the following: mv -r "$1" /mnt/thawed/ However, if I use this for multiple indexes, I need to move it to the path for the index: /mnt/thawed/indexname/ How can I genericize this? I mean, I can set an argument, but can I detect the index or the thawedPath easily? ... View more

The following tells me how many events I'm indexing every 5 minutes. index="_internal" group="thruput" | bucket _time span=5m | timechart span=5m sum(instantaneous_eps) as ieps I'd really like to finish this with a threshold, like this: index="_internal" group="thruput" | bucket _time span=5m | timechart span=5m sum(instantaneous_eps) as ieps | ieps < 5000 ..except that doesn't work. ... View more

The following tells me how many events I'm indexing every 5 minutes. index="_internal" group="thruput" | bucket _time span=5m | timechart span=5m sum(instantaneous_eps) The graph isn't accurate, it shows the number of results in the bucket. It'd be nice if that was pretty. More important, it means I have a large number of rows to look through. What I'd really like is to have a per-day min/max/avg of these bucketed events. In other words, summarize the 5-minute span of events into three per-day numbers. Can someone show off their advanced Splunk skills on this for me? ... View more

I'm watching a directory. Let's say it is /foo. The files are in subdirectories: /foo/archive/2010-11/ /foo/archive/2010-10/ /foo/archive/2010-09/ It doesn't appear Splunk is looking recursively to find those subdirectories. Do I need to add every individual month to Splunk? What are my options? One thought is I could modify the archive script to put a copy of the file in the spool directory, but that means the index isn't "hard set" like it is on that monitored directory. What else? Perhaps I could have Splunk watch /foo/incoming, I'll copy it there and Splunk could read and delete it from that directory? I think "..." is what I need for recursion. The inputs.conf doesn't make it clear- would the following monitor work: [monitor:///foo/...] Or this? [monitor:///foo/.../*] ... View more

There must be an easy way to fire a single message over UDP to a splunk forwarder/server. "logger" nearly does it. I want to do something like this: testlogger -h host -p port "message to log in Splunk" Let me specify- Unix commandline. I suppose I could write a few lines of code, but am I the first person to want this? ... View more