This is bug SPL-31616 fixed in 4.1.4. ... View more

I assume that you're using the REST API like: GET /services/search/jobs/<sid>/results?output_mode=csv If this is the case, there are two arguments to add, field_list and count : GET /services/search/jobs/<sid>/results?output_mode=csv&field_list=<comma separated field list>&count=0 If you want to page through the results for efficiency, you can use the offset argument in addition to the count argument and make several requests. ... View more

Note that if you search for this field alone, because it's marked as a non-indexed value, Splunk will perform a full table scan to find matches. To get around this performance issue, you could extract the field at index time, set up a lookup table that maps all sources to your fields, or set up a set of eventtypes. ... View more

Your regex is designed to match the 6th non-whitespace item in the string, but the first five items are allowed to be degenerate (because of the * after the [^ ]). Occasionally it will find what appears to be the fifth word, because it will accept as the second item the empty string between the two spaces between the month and the day. You want a regex like: ^(?:\S+\s+){4}(?<process>\S+) ... View more

If you have a good partition on these records, you can delete them that way. For example "index=summary source=my_search a* | delete", and repeat for "b*", "c*", etc. As far as retrieving more than 500k records for the same second, we could either consider running the summary more often, or tweaking the timestamps so that these >500k events are spread over several seconds and not just one. I am concerned, however, that if we have this many records for one day, the summary will be suboptimal. We may want to just consider scheduling a nightly search to report over the previous 30 days. ... View more

Lowell, sorry about this oversight. We have fixed exporttool to restore backwards compatibility for calling arguments. This will ship in 4.1.5. ... View more

Yes, this can be done using a custom search script and one of the many Python modules that can compare strings. You can take a look at http://stackoverflow.com/questions/682367/good-python-modules-for-fuzzy-string-comparison which discusses using the Levenshtein distance as a measure. With more detail about your use case, I could suggest how to structure a search and custom command, but this should be enough to start with. ... View more

First, for your regex, it should consume the "," as the TIME_PREFIX must take you up to the time format itself. However, the more real question is why the lines are still sticking together even if you set SHOULD_LINEMERGE = false. Are you certain that this is set for the relevant stanza? If it is, then this configuration is just not getting referenced. ... View more

Timechart has a couple of helpful time-based aggregators: per_second() , per_minute() , per_hour() that sum the field in parenthesis and divide by the number of seconds, minutes or hours in each timechart bin. Since this requires something to count, you could write your search as: ... | eval weight = 5 | timechart per_minute(weight) Alternately you could do after-the-timechart processing on the _span field added by timechart like the following, which normalizes timechart buckets that are larger than 5 minutes (300s): ... | timechart count | eval count = count*300/_span ... View more

As gkanapathy said, this is not possible with CSV lookup tables, but is feasible with external lookups. That said, I think that eventtypes are probably the right knowledge primitive to use with version 4.1 (and 4.2, when it comes out) to solve this problem. For example, in our internal web analytics app, we use eventtypes to "classify" events as being from bots, browsers, internal IPs and in turn pageviews are driven by eventtypes. You can configure eventtypes.conf to enumerate the regions like: [location-australia-sydney] search = Subnet=10.50.6.0/24 [location-australia-melbourne] search = Subnet=10.50.7.0/24 To retrieve, you can search for eventtype=location-australia-sydney . If you want all of Australia, you can search for eventtype=location-australia-* . Since these are eventtypes, events are tagged with these for later filtering or reporting. If you just want to report on these locations, you can add to your search: ... | eval location = mvfilter(eventtype LIKE "location-%") | ... ... View more

You should not modify files directly from your module. It's a better idea to use splunkd's REST API to modify files. Specifically, if you POST to https://server:port/servicesNS/<user>/<app>/admin/conf-<conf-file>; , you'll create entries in $SPLUNK_HOME/etc/users/<user>/<app>/local/<conf-file>.conf . You can create a stanza named foo with a line key=value by POSTing name=foo&key=value . You can read this stanza at servicesNS/<user>/<app>/admin/conf-<conf-file>/<stanza> . Helpers for reading and writing from these can be found in entities.py which ships with Splunk. Note that this method is not officially supported. ... View more

The best technique for TIME_PREFIX in this case is the positive lookahead assertion. For example: "TIME_PREFIX = ,(?=\d+/\d+/\d{4} \d\d:\d\d)". In other terms, take us to just before something that looks like the date. ... View more

Note that we're not storing a daily count summary. We're storing, in the most condensed fashion possible, the list of values for each day. The second search has the net effect of performing that dedup and the stats count at the end counts the deduplicated values. ... View more

This is certainly a little tricky but is definitely doable. Below is a search sample. Assume that you have a field called "metrics" with the above data. ... | rex field=metrics "(?<metric_triple>\w+ \w+ \w+)" max_match=100 | fields - metrics | mvexpand metric_triple | rex field=metric_triple "(?<Devicename>\w+) (?<Attribute>\w+) (?<Value>\w+)" | fields - metric_triple The next lines pull out triples of strings, we expand each event into one per triple and finally pull out the three fields for each metric. Interested readers can simulate this data with the lines: | stats count as host | eval host = "foo" | eval metrics = "L2cache0 size 0 cd0 audio_supported yes cd0 cdda_supported yes cd0 data_verify enable cd0 prevent_eject yes en0 alias4 en0 alias6 en0 arp on en0 authority en0 broadcast en0 mtu 1500 en0 netaddr 157.191.125.12 en0" ... View more

Gerald, not needed with the stats. In fact that will make it slower. ... View more

You don't really want to use the timechart command if you don't want one row in your table per timespan. I think that you want a search like: source="/var/logs/current/juniper" Packet drop in Ichip pktwr | rex "(?<fpc>fpc\d+) (?<ichip>ICHIP\(\d+\)):Packet drop in Ichip pktwr,rate: %\S+: \d+, total: (?<err>\d+)" | stats range(err) as pktwr_drops by host, fpc, ichip | eval rate = pktwr_drops/720 | sort - rate You can even be more clever by using the addinfo command to find the timebounds of the search and use that instead of the constant 720: source="/var/logs/current/juniper" Packet drop in Ichip pktwr | rex "(?<fpc>fpc\d+) (?<ichip>ICHIP\(\d+\)):Packet drop in Ichip pktwr,rate: %\S+: \d+, total: (?<err>\d+)" | stats range(err) as pktwr_drops by host, fpc, ichip | addinfo | eval rate = pktwr_drops/(info_max_time-info_min_time) | fields - info* | sort - rate ... View more