Custom search command (v2/chunked): A lot of time passes between prepare() and generate()

[spunk_enthusias]

I am building a GeneratingCommand and even in the most basic version a lot of time passes between the invocation of prepare() (called after about 50ms after start) and generate() (called about 170ms after start).

We plan to invoke this command frequently, so this gap matters. How can I reduce it?

[spunk_enthusias]

Now here's an interesting tidbit: Where a bare-bones Python script using Splunklib takes 170ms to dispatch.evaluate, a dbxquery from DB Connect takes 80ms for a whole query! And that's also with a chunked python3 search command.

So what's different? It doesn't use splunklib! Or at least not the Python version. The Python script acting as the bridge simply forwards the search protocol messages to a TCP server which likely uses the Java implementation.

I've said before that splunklib could use some attention - case in point.