There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds. 1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing). OR 2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result) ... View more

You can configure Splunk to assign the hostname in a few different ways. They're all documented here: http://docs.splunk.com/Documentation/Splunk/5.0/Data/Aboutdefaultfields In your case it makes sense to specify a host for the input looking at the directory, and to use a segment in path value of 4. How Splunk assigns the host value If no other host rules are specified for a source, Splunk assigns host a default value that applies to all data coming from inputs on a given Splunk server. The default host value is the hostname or IP address of the network host. When Splunk is running on the server where the event occurred (which is the most common case) this is correct and no manual intervention is required. For more information, see "Set a default host for a Splunk server" in this manual. Set a default host for a file or directory input If you are running Splunk on a central log archive, or you are working with files forwarded from other hosts in your environment, you may need to override the default host assignment for events coming from particular inputs. There are two methods for assigning a host value to data received through a particular input. You can define a static host value for all data coming through a specific input, or you can have Splunk dynamically assign a host value to a portion of the path or filename of the source. The latter method can be helpful when you have a directory structure that segregates each host's log archive in a different subdirectory. For more information, see "Set a default host for a file or directory input" in this manual. Override default host values based on event data You may have a situation that requires you to override host values based on event data. For example, if you work in a centralized log server environment, you may have several host servers that feed into that main log server. The central log server is called the reporting host. The system where the event occurred is called the originating host (or just the host). In these cases you need to define rules that override the automatic host assignments for events received from that centralized log host and replace them with distinct originating host values. For more information, see "Override default host values based on event data" in this manual. Tag host values Tag host values to aid in the execution of robust searches. Tags enable you to cluster groups of hosts into useful, searchable categories. ... View more

In 4.1 Splunk introduced a "developer services" page available off of Splunkweb: http://<splunk-web-ip:port>/info This includes a "Splunkweb endpoint reference" as well as an "EAI object refresh" link that will refresh the following objects in one swoop: Refreshing admin/conf-times Refreshing data/ui/manager Refreshing data/ui/nav Refreshing data/ui/views Refreshing admin/cooked Refreshing admin/deploymentserver Refreshing admin/deploymenttenants Refreshing admin/fifo Refreshing admin/monitor Refreshing admin/raw Refreshing admin/script Refreshing admin/ssl Refreshing admin/udp ... View more

The fields are h0rked because the W3C ELFF format introduces spaces into the timestamp, and the space character is being used as the delimiter for field extraction. You would need to modify the list of expected fields in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/transforms.conf, here: [delimExtractions] DELIMS=" " FIELDS="date","time","time_taken","dvc_ip","user","user_group","x_exception_id","filter_result","category","http_referrer","holder","http_response","action","http_method","http_content_type","uri_scheme","dest_host","dest_port","uri_path","uri_query","uri_extension","http_user_agent","src_ip","sc_bytes","cs_bytes","x_virus_id" In addition, if you're changing the format of the timestamp you'll probably also have to change the following line in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/props.conf: TIME_FORMAT=%Y-%m-%d %T For both changes, the caveat applies of not updating default configs, or any upgrade will revert the changes. The practice is to create the config in local/ and only copy over the settings that are being changed. Lastly, I would expect that all of this should be moot, since Splunk normalizes any timestamp in the events and stores it internally as UTC anyway. When a user searches for data, the time is then converted to the localtime of the browser. Perhaps I'm not understanding the original issue that prompted you to change to W3C? ... View more

date_hour is a default field. You could do: ... earliest=-7d | stats count by date_hour This is something that you probably don't want to do ad-hoc. I recommend summary indexing: http://docs.splunk.com/Documentation/Splunk/5.0/Knowledge/Usesummaryindexing ... View more

If the output queue fills up, all preceding Splunk processors will block. This means if you're monitoring a file or directory, the tailing processor will block and stop moving the pointers into each file. Once the indexer is up and the output queue empties, the tailing processor will unblock and the pointer will eventually catch up. You shouldn't lose data, unless the outage is so long that the file gets rolled or deleted. If you have network inputs, no such luck. ... View more