Is it possible to have another value for "blob_mode" than "append" or to index the blob one time and after do not check changes?
We write in a blob one time, and after that, we do not make changes. However, Splunk continues to check if there's a modification.
We have more than 100 000 blobs and now it's affecting performance having to check all these blobs.
Hi nbouchia,
thanks for the proposal, actually the add-on DO NOT touch the file again once it's indexed and have no further changes.
I think it's related tot he large count of blob files, but would you please provide with more detail about the performance issue you encountered?
We have an application that's write in a different blob at each instantiation. We have around 200 000 blobs in the container.
When a blob is created after it doesn't change.
With the add-on we pull the container that contains all these blob. We configure the add-on to pull every 5mn.
I thinks our problem performance occured when the add-on test the last modified time for all blobs to see if there's new data to index.
Hi nbouchia
thanks for the reply, may i know the the performance as it is in your env? say, how much delay or lag? OTOH, if you could provide some screenshot as well diag files (by reference Splunk diag), that will be very helpful. Thanks!
Hello
We have one server (indexer+search head in the same server).
Initialy, the delay was put to 60sec but now we put it to 10mn.
And now, we archive the blobs, so we don't have the same problem but with time it will happen again.
We showed the problem with a lot of I/O in our system and some crashed.
