Django ORM leaves DB connections open

[ciur]

Django ORM leaves open database connections in IDLE state for following REST API endpoints:

- GET /api/users/me
- GET /api/nodes/<id>
- GET /api/thumbnails/<id>
- GET /api/documents/<id>
- GET /api/pages/<id>/jpg
- GET /api/pages/<id>/svg

As result connections pile-up and when reaching specific limit, e.g. 100, 50, 25, depending on DB config, the app stops working as DB refuses all further connections. Same behavior was observed for PostgreSQL 16.1 and MariaDB 11.2.

Expected behavior: DB connections should be released when not used; or number of DB connections should remain stable when connection pooling is employed.

Info:

• Browser: any
• PostgreSQL 16.1
• Papermerge 3.0

[ciur]

First PR which fixes couple of critical endpoints: papermerge/papermerge-core#292

[ciur]

PR 2: papermerge/papermerge-core#294
PR 3: papermerge/papermerge-core#295

The solution is to replace Django ORM with SQLAlchemy. Of course ORM replacement it not matter of couple of weeks.
This ticket is for fixing the most critical parts so that app is usable in production.

This fix will be shipped in 3.0.1 release.