https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652890#M225611 <P>You can use the collect command and specify the index and sourcetype (although I think this may add to your license usage, whereas if you default the source type it becomes stash which I think avoids additional license usage, but you should check that). You might want to check back with your admins as to why you can't use a different index (as this also gives you the potential for different retention periods and storage options).</P> Wed, 02 Aug 2023 12:01:05 GMT ITWhisperer 2023-08-02T12:01:05Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652884#M225605 <P>Hello Splunk Experts,</P> <P>I'm searching for ERRORS and WARN in the application from different servers and trying to collect these log lines to a stored area(Summary Index - may be Sourcetype) to avoid searching again &amp; again on a huge volume of data. I don't want to use lookup because of the data volume. What is the procedure to get this done. Could someone please assist. Thanks in advance!!</P> Wed, 02 Aug 2023 20:27:06 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652884#M225605 Thulasinathan_M 2023-08-02T20:27:06Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652888#M225609 <P>You need to create (or ask your administrators to create) an index. You can then schedule a report to extract a subset of the results and add them to the summary index with the collect command. Depending on what you want in your summary index and how you are going to process it afterwards, you may have to consider making the update to the summary index idempotent to avoid adding the same information to the summary index multiple times.</P> Wed, 02 Aug 2023 11:29:05 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652888#M225609 ITWhisperer 2023-08-02T11:29:05Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652889#M225610 <P>Hi&nbsp;<a href="https://community.splunk.com/t5/user/viewprofilepage/user-id/225168">@ITWhisperer</a>&nbsp;. Thanks for the info, using same index would be fine just I want it to get written in either a new file in new soucetype or a new log file on same sourcetype. I've checked with our Admins, they're advising to do it application level. So just trying to understand why it's not feasible from splunk.</P> Wed, 02 Aug 2023 11:51:46 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652889#M225610 Thulasinathan_M 2023-08-02T11:51:46Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652890#M225611 <P>You can use the collect command and specify the index and sourcetype (although I think this may add to your license usage, whereas if you default the source type it becomes stash which I think avoids additional license usage, but you should check that). You might want to check back with your admins as to why you can't use a different index (as this also gives you the potential for different retention periods and storage options).</P> Wed, 02 Aug 2023 12:01:05 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/652890#M225611 ITWhisperer 2023-08-02T12:01:05Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/653026#M225646 <P>Hi&nbsp;<a href="https://community.splunk.com/t5/user/viewprofilepage/user-id/225168">@ITWhisperer</a>&nbsp;,</P><P>Thanks again!! License usage is not a problem in our case. I just need these to stored in a dedicated path, instead of storing in stash. Is there any article/ links I could refere to, could you please kindly help!!</P> Thu, 03 Aug 2023 09:25:48 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/653026#M225646 Thulasinathan_M 2023-08-03T09:25:48Z https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/653036#M225651 <P><A href="https://docs.splunk.com/Documentation/Splunk/9.1.0/SearchReference/Collect" target="_blank">collect - Splunk Documentation</A></P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="ITWhisperer_0-1691055931129.png" style="width: 400px;"><img src="https://community.splunk.com/t5/image/serverpage/image-id/26622i1DFE3F8794C662AE/image-size/medium?v=v2&amp;px=400" role="button" title="ITWhisperer_0-1691055931129.png" alt="ITWhisperer_0-1691055931129.png" /></span></P><P>&nbsp;</P> Thu, 03 Aug 2023 09:45:38 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collect-a-huge-volume-of-data/m-p/653036#M225651 ITWhisperer 2023-08-03T09:45:38Z