[cpp] Allow starting lookups without blocking

[fresh-borzoni]

closes #4558

A C++ lookup blocks until its result arrives, and each lookup waits for the 100 ms batch window, so one thread gets about 10 lookups per second. In Java, Rust and Python lookup() returns a future, so one thread can keep many lookups in flight.

This adds Lookuper::Lookup(pk_row, PendingLookup&) and PrefixLookuper::PrefixLookup(prefix_row, PendingPrefixLookup&), which start the lookup and return at once, the same shape as Append(row, WriteResult&). Wait(out) collects the result. Wait(out, timeout_ms) returns the retriable REQUEST_TIME_OUT if the result isn't there yet and leaves the lookup running, so it can be waited for again.
Destroying the handle before Wait returns abandons the lookup.

[fresh-borzoni]

@leekeiabstraction @charlesdong1991 PTAL 🙏

[naivedogger]

Hi @fresh-borzoni, thanks for working on this.
This feels similar to the Append(row, WriteResult&) model we had before adding callbacks: callers need to keep a handle for each request and wait on them individually. Could we consider LookupAsync(key, request_id) plus a Poll(timeout) that returns a batch of completed lookups, each carrying its request ID, status and result, similar to Kafka consumer polling? That would let callers process whichever requests have completed without managing a collection of pending handles or waiting behind a slow request when iterating through them. I'd lean toward this for the async API while keeping the existing blocking lookup. What do you think?

[fresh-borzoni]

Thanks for the review @naivedogger!

I'd keep the handle as the base async API: it's the future Java, Rust and Python already return, and it covers the simple start-now, wait-later case without a polling loop or request IDs.
Poll fits fan-out where you want completion order, so shall we add it as a follow-up next to the handle? It would need a bound on completed lookups nobody has polled. WDYT?

[naivedogger]

@fresh-borzoni Keeping the handle API here and adding polling in a follow-up sounds good to me. Overall, LGTM, with just one minor comment on the timeout documentation.

[fresh-borzoni]

@naivedogger Thank you for the review, addressed, PTAL

Filed #4705 as a followup

[naivedogger]

@fresh-borzoni Thanks for the update. LGTM.