https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377696#M3219 <P>The issue turned out to be s3 compatibility storage does not support a delimiter "string" but just a delimiter character, so Splunk need to apply configuration change to disable delimiter. </P> <P>Due to this change in delimiter within Splunk Code will have to parse through a lot more object names to find buckets, and it will be inefficient.</P> <P>So, Splunk will end up disabling the delimiter, which will mean the object listings will be inefficient. This is one of the limitations of not having full aws s3 compatibility.</P> <P>In server.conf </P> <PRE><CODE>remote.s3.use_delimiter = true | false * Optional. * Specifies whether a delimiter (currently "guidSplunk") should be used to list the objects that are present on the remote storage. * A delimiter groups objects that have the same delimiter value so that the listing process can be more efficient as it does not need to report similar objects. * Defaults to: true </CODE></PRE> Wed, 19 Dec 2018 05:57:44 GMT rbal_splunk 2018-12-19T05:57:44Z https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377695#M3218 <P>Our indexers are configured to use s3 compatibility remotepath, ans were seeing lots of 400 status code returned when Splunk makes calls to S3<BR /> The isolated all URI contains a parameter "&amp;delimiter=guidSplunk". URI with parameters that does not have the parameter "delimiter=guidSplunk" all have the paramter "versions" parameter no value and those succeeds. For example:</P> <P>Failing URI:<BR /> /secsplunk-idx-sysmon?max-keys=1000&amp;prefix=sysmon%2Fdma%2F55%2Fe5%2F1595~BED2107F-430E-49FC-8449-949FA7F70D51&amp;delimiter=guidSplunk</P> <P>Succedding URI:<BR /> /secsplunk-idx-sysmon?versions&amp;max-keys=1000&amp;prefix=sysmon%2Fdb%2F55%2Fe5%2F1595~BED2107F-430E-49FC-8449-949FA7F70D51%2FguidSplunk-BED2107F-430E-49FC-8449-949FA7F70D51%2Frawdata%2Fslicesv2.dat</P> Wed, 19 Dec 2018 05:51:20 GMT https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377695#M3218 rbal_splunk 2018-12-19T05:51:20Z https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377696#M3219 <P>The issue turned out to be s3 compatibility storage does not support a delimiter "string" but just a delimiter character, so Splunk need to apply configuration change to disable delimiter. </P> <P>Due to this change in delimiter within Splunk Code will have to parse through a lot more object names to find buckets, and it will be inefficient.</P> <P>So, Splunk will end up disabling the delimiter, which will mean the object listings will be inefficient. This is one of the limitations of not having full aws s3 compatibility.</P> <P>In server.conf </P> <PRE><CODE>remote.s3.use_delimiter = true | false * Optional. * Specifies whether a delimiter (currently "guidSplunk") should be used to list the objects that are present on the remote storage. * A delimiter groups objects that have the same delimiter value so that the listing process can be more efficient as it does not need to report similar objects. * Defaults to: true </CODE></PRE> Wed, 19 Dec 2018 05:57:44 GMT https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377696#M3219 rbal_splunk 2018-12-19T05:57:44Z https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377697#M3220 <P>The resolution was &gt; string delimiters are working but appear that there is a bug opened for this issue.<BR /> <A href="http://tracker.ceph.com/issues/24821">http://tracker.ceph.com/issues/24821</A></P> Sat, 19 Jan 2019 06:11:46 GMT https://community.splunk.com/t5/Knowledge-Management/SmartStore-Splunk-configured-with-s3-compatibility-smartstore/m-p/377697#M3220 rbal_splunk 2019-01-19T06:11:46Z