On a Windows 2012 Server the daily IIS log is held open and sits at "0" bytes in size throughout the day. It appears to be only written to disk at the daily rollover time, typically 10am at which time is becomes "x" bytes in size. Does this mean that Splunk will only receive the total daily events at this time. i.e. delayed by 24 hours. If this is the case how does Splunk Universal Forwarder send the traffic? does it send the entire file in 1 go or does it potentially send the entire file line by line until complete, thus possibly sending large amounts of network traffic at rollover time?
It is not one line at a time, but it sends everything.
Network traffic depends on ”maxKBps” setting.
maxKBps = <integer>
* The maximum speed, in kilobytes per second, that incoming data is
processed through the thruput processor in the ingestion pipeline.
* To control the CPU load while indexing, use this setting to throttle
the number of events this indexer processes to the rate (in
kilobytes per second) that you specify.
* There is no guarantee that the thruput processor
will always process less than the number of kilobytes per
second that you specify with this setting. The status of
earlier processing queues in the pipeline can cause
temporary bursts of network activity that exceed what
is configured in the setting.
* The setting does not limit the amount of data that is
written to the network from the tcpoutput processor, such
as what happens when a universal forwarder sends data to
* The thruput processor applies the 'maxKBps' setting for each
ingestion pipeline. If you configure multiple ingestion
pipelines, the processor multiplies the 'maxKBps' value
by the number of ingestion pipelines that you have
* For more information about multiple ingestion pipelines, see
the 'parallelIngestionPipelines' setting in the
* Default (Splunk Enterprise): 0 (unlimited)
* Default (Splunk Universal Forwarder): 256