https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161485#M32729 <P>Thanks Cary <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P> Sat, 28 Feb 2015 17:19:45 GMT garimayadav 2015-02-28T17:19:45Z https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161483#M32727 <P>What is the frequency with which logs are read in Splunk? Does delay in seeing recent log details in Splunk related to poor indexing?<BR /> And if indexing is perfectly fine, then what should be the ideal time after which details of a new request must be visible in Splunk?</P> Sat, 28 Feb 2015 11:29:54 GMT https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161483#M32727 garimayadav 2015-02-28T11:29:54Z https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161484#M32728 <P>From my understanding, there is no "frequency." If the file changes, it starts forwarding the new data. There are technically several ways to do this, I don't know which one Splunk uses, but I have used a few throughout my programming career.</P> <P>How soon should you see indexed data in Splunk? That depends on the processing of the data at index time. If not much is being looked at, you can have data visible sub-second after it is placed in a file on a forwarder, if all the stars align properly. We index about 350GB/day and see most of our data within 10 seconds with most of that under 5 seconds. I have had "Last 5 seconds" searches that fill up even the last 1 second slot with almost as many events as the other 4 slots.</P> <P>If your indexing is putting structure on the data, or the indexing has to search hard to find the timestamp, or your network is slow, or the indexing hardware is slow (especially the disks), these things can lead to slow indexing of the data.</P> Sat, 28 Feb 2015 16:07:21 GMT https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161484#M32728 cpetterborg 2015-02-28T16:07:21Z https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161485#M32729 <P>Thanks Cary <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P> Sat, 28 Feb 2015 17:19:45 GMT https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161485#M32729 garimayadav 2015-02-28T17:19:45Z https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161486#M32730 <P>Hi garimayadav,</P> <P>an universal forwarder will poll / check the files once a second.</P> <P>cheers, MuS</P> Sun, 01 Mar 2015 16:17:41 GMT https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161486#M32730 MuS 2015-03-01T16:17:41Z https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161487#M32731 <P>Hi garimayadav -</P> <P>Questions asked by you purely depends upon data volume/day, underlying hardware, deployment architecture, network latency. Following links will address your queries. Please have a look. </P> <P><A href="http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Forwarder-to-indexerratios">http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Forwarder-to-indexerratios</A><BR /> <A href="http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Referencehardware">http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Referencehardware</A><BR /> <A href="http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Performancechecklist">http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Performancechecklist</A><BR /> <A href="http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Summaryofperformancerecommendations">http://docs.splunk.com/Documentation/Splunk/6.2.2/Capacity/Summaryofperformancerecommendations</A></P> Mon, 02 Mar 2015 05:53:31 GMT https://community.splunk.com/t5/Getting-Data-In/What-is-the-frequency-with-which-logs-are-read-in-Splunk/m-p/161487#M32731 satishsdange 2015-03-02T05:53:31Z