rhizome RX can fail due to locked database

[gardners]

... resulting in repeated rhizome fetches, which may also fail, and generally clog up all available bandwidth without doing anything really useful.

Happening quite a bit at LCA2013 with the congested WiFi. Not sure how to reproduce in the lab.

[quixotique]

What is probably needed here is a change in rhizome_fetch.c to allow rhizome_write_content() to suspend the (pseudo) thread until the db write succeeds, by scheduling a short alarm if the SQlite write operation times out due to a lock. That will prevent the Rhizome fetch logic from closing the slot and re-starting the fetch from scratch, and will also slow the fetch rate down to the rate at which the local SQLite db can accept content (given its lock density).

[lakeman]

The impact of issue has been greatly reduced. We now pass the full manifest to batphone via the monitor interface, so we don't need to hit the database for that. And we have re-implemented the MeshMS API using restful HTTP (See 73e1a70, 8c71c34), so we don't need to hit the database directly. Shortly we should be able to perform all other rhizome commands via HTTP, reducing the impact still further.