Search is index="_internal" source="*metrics.log" group="queue" | timechart perc90(current_size) by name
Results are:
group=queue, name=parsingqueue, blocked!!=true, max_size=1000, filled_count=15, empty_count=0, current_size=1000, largest_size=1000, smallest_size=996
group=queue, name=aggqueue, blocked!!=true, max_size=1000, filled_count=31, empty_count=0, current_size=1000, largest_size=1000, smallest_size=930
Are you actually experiencing problems with indexing throughput?
Increasing the length of the queue will probably not help. A constantly filled queue indicates that the processing that takes place on it is unable to keep up with the incoming work. Increasing the queue may give you a little room if this happens because your data comes in small bursts. If you are not experiencing indexing throughput problems, there's nothing you need to do.
If you are experiencing indexing throughput problems, there are a few options. Among them:
Are you actually experiencing problems with indexing throughput?
Increasing the length of the queue will probably not help. A constantly filled queue indicates that the processing that takes place on it is unable to keep up with the incoming work. Increasing the queue may give you a little room if this happens because your data comes in small bursts. If you are not experiencing indexing throughput problems, there's nothing you need to do.
If you are experiencing indexing throughput problems, there are a few options. Among them:
you can ignore pulldown. It just controls whether the sourcetype appears in the GUI list. the other problem may just be that you need a faster machine or faster disk.
Indexing is very slow - added 250 mb to indices - helped some - going to the customized time stamping formats next due to mixed windows, sourcefire, and cisco data - everything is single line coming from snare and syslog so will turn on Should_linemerge = false - regexes are spot on .. and only as long as I need to pull fields from .. thanks for the help will check back. - What is the pulldown value all about - noticed it in the props.conf in default - should it be added to the local props.conf ?