Another option would be to send the data in on 514 and then use some other criteria to override the sourcetype of the data. It is a normal problem for many different types of devices to send syslog on port 514 which you then want to separate into different sourcetypes. Here is how you do it: Set up a generic UDP listener for 514 into a generic syslog source type (make sure your reverse DNS is working) [udp://514] connection_host = dns sourcetype = syslog index = myindex Set up a sourcetype override in transforms.conf similar to the following [new_sourcetype] SOURCE_KEY = MetaData:Host DEST_KEY = MetaData:Sourcetype REGEX = (?i)(myhostname) FORMAT = sourcetype::new_syslog In your props.conf link the syslog sourcetype with the tranform [syslog] TRANSFORMS-sourcetype_override = new_sourcetype LOOKUP-rdns = dnslookup clientip AS host OUTPUT clienthost AS dnsHostname ... View more

Would it be based on endEpoch rather than modTime? I notice that these two buckets have higher endEpoch values than others. ... View more

I did see this post in my research. I'm not entirely sure that it would make a difference. The space limit for warm has already been reached and cold is also full so buckets from only 4 months ago are being rolled from cold to frozen, so fiddling with the limits will probably only cause newer bucket to be rotated. If it was going to move these based on age wouldn't it have done so already? ... View more

I tried completely changing the export script to use key/value pairs instead of CSV, set up a new monitor on a different folder into a temporary index and have observed the same issue. It eliminates the specific parts of the config on the indexer side of the props and tranforms section relating specifically to this source type as being the issue. What I can't understand is that I have a second export from the same forwarder using a different select statement which isn't affected by this issue. ... View more

Note that the spaces in < SOURCE > are not in the actual inputs.conf. I have inserted them here because the editor was removing the statement without them. ... View more

Pavan's response was correct. gokadroid was not. ... View more

"minus sign too close to MegaBytes" is not the answer. No space is required between the minus and the sort field. It works with or without the space between the minus and the sort field. ... View more

You are correct. Turns out an en-dash crept into my search. PS C:\> [int][char]'–' 8211 PS C:\> [int][char]'-' 45 ... View more

Actually that is wrong. It turns out an en-dash managed to creep into my search from copy/paste. You don't need a space between the minus and the sort field. ... View more

Sound plausible. From the [scheduler] section: max_searches_perc: defaults to 50. I ended up changing the rt search multiplier from 1 to 2. I can't find any good data on what value it is safe to set this to. Some people have said 4 and one person set it to 8. Setting it to 2 got me out of trouble in this case. I wish the documentation was clearer on this. ... View more

Correct. On the same line. ... View more

I'm 100% sure about the number of cores. Checked through Task Manager on Windows, plus I administer the VM it is running on and it has 2 x sockets with 8 cores each presented to it. ... View more

If I sort by -active_realtime_searches then I get 13 as the max number of realtime searches in the last 24 hour period and 15 for historical. I do get a row of 15 historical and 11 reatime ... View more

Even for the last 4 hours that returns 240,000+ results and they seem to read 0,0. ... View more

And it is a single-server instance, so indexer and search head all on one server. Running Splunk 6.2.6 ... View more

From running splunk.exe show config limits I get the following for the [search] stanza. To my knowledge these are all defaults: [search] allow_batch_mode=true allow_inexact_metasearch=false base_max_searches=6 batch_retry_max_interval=300 batch_retry_min_interval=5 batch_retry_scaling=1.5 batch_search_max_index_values=10000000 batch_wait_after_end=900 cache_ttl=300 chunk_multiplier=5 default_allow_queue=true default_save_ttl=604800 dispatch_dir_warning_size=2000 dispatch_quota_retry=4 dispatch_quota_sleep_ms=100 enable_history=true failed_job_ttl=86400 fetch_remote_search_log=disabledSavedSearches fieldstats_update_freq=0 fieldstats_update_maxperiod=60 load_remote_bundles=false long_search_threshold=2 max_chunk_queue_size=1000000 max_combiner_memevents=50000 max_count=500000 max_history_length=1000 max_id_length=150 max_macro_depth=100 max_rawsize_perchunk=100000000 max_results_perchunk=2500 max_rt_search_multiplier=1 max_searches_per_cpu=1 max_tolerable_skew=60 max_workers_searchparser=5 min_freq=0.01 min_prefix_len=1 min_results_perchunk=100 multi_threaded_setup=0 preview_duty_cycle=0.25 queued_job_check_freq=1 realtime_buffer=10000 reduce_duty_cycle=0.25 reduce_freq=10 remote_timeline=true remote_timeline_connection_timeout=5 remote_timeline_fetchall=1 remote_timeline_min_peers=1 remote_timeline_receive_timeout=10 remote_timeline_send_timeout=10 remote_timeline_touchperiod=300 remote_ttl=600 replication_file_ttl=600 replication_period_sec=60 result_queue_max_size=100000000 results_queue_min_size=10 rr_max_sleep_ms=1000 rr_min_sleep_ms=10 rr_sleep_factor=2 search_process_mode=auto stack_size=4194304 status_buckets=0 status_cache_size=10000 summary_mode=all sync_bundle_replication=auto target_time_perchunk=2000 track_indextime_range=true truncate_report=false ttl=600 write_multifile_results_out=true ... View more

But why is it 11? According to the doco it should be 22. ... View more

Hi, Thanks for the reply. I should mention that I've done some "All Time" searches against this host in case the events were showing up in the future with no luck. As you say, being a DC, I'd have other problems with time sync. The server is definitely in a different time zone, but I have two servers at the site and it is only the DC that is not forwarding Windows Event logs and they are both covered by the same entry in the Splunk config for time zone adjustment. We are forwarding Splunkd logs and I checked for DateParserVerbose errors but nothing came up. In fact the only ERROR present is one about it not being able to locate the PDC emulator, but every DC has that error. ... View more

Did you ever get a resolution? I've got the same issue with 2 forwarders out of MANY with identical configuration. ... View more

Good answer. Thanks. We are using REPORT-, so it is all good. ... View more