Synapse 1.12.0 requires sqlite3 even if you're using postgres

[richvdh]

Turns out that some distributions don't ship the sqlite3 library as part of their python:

(env) $ synctl start
Starting ...
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/app/homeserver.py", line 53, in <module>
    from synapse.federation.transport.server import TransportLayerServer
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/federation/transport/server.py", line 48, in <module>
    from synapse.server import HomeServer
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/server.py", line 31, in <module>
    from synapse.api.filtering import Filtering
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/api/filtering.py", line 30, in <module>
    from synapse.storage.presence import UserPresenceState
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/__init__.py", line 30, in <module>
    from synapse.storage.data_stores import DataStores
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/data_stores/__init__.py", line 18, in <module>
    from synapse.storage.data_stores.state import StateGroupDataStore
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/data_stores/state/__init__.py", line 16, in <module>
    from synapse.storage.data_stores.state.store import StateGroupDataStore  # noqa: F401
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/data_stores/state/store.py", line 26, in <module>
    from synapse.storage._base import SQLBaseStore
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/_base.py", line 27, in <module>
    from synapse.storage.database import LoggingTransaction  # noqa: F401
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/database.py", line 38, in <module>
    from synapse.storage.background_updates import BackgroundUpdater
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/background_updates.py", line 25, in <module>
    from . import engines
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/engines/__init__.py", line 19, in <module>
    from .sqlite import Sqlite3Engine
  File "/var/synapse/synapse/env/lib/python3.7/site-packages/synapse/storage/engines/sqlite.py", line 15, in <module>
    import sqlite3
  File "/usr/local/lib/python3.7/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/local/lib/python3.7/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'
error starting (exit code: 1); see above for logs

[richvdh]

Introduced in #6987 I think

[anoadragon453]

Does

synapse/synapse/storage/engines/sqlite.py

Line 15 in 4170b2e

import sqlite3

need to be moved?

[richvdh]

yes

[richvdh]

or rather, it probably needs to be be made conditional and only used when type-checking

[matoro]

Just tried to upgrade and ran into this. Will this be fixed in a minor update to the 1.12 release?

[richvdh]

workaround is to install whatever package contains the sqlite library on your OS.

I suspect it only affects FreeBSD, since linux distros ship sqlite as part of their stock python (which is why we didn't notice this sooner).

[vsatmydynipnet]

Thanks to richvdh for mentioning this in Synapse room. I shot my server too and was unable install sqlite 3 stuff. A minor update would be great, then I can also update Twister lib. Thank you.

[anoadragon453]

Hmm, I gave an initial stab at this in #7143 but don't seem to be able to come up with a solution that solves the problem, appeases mypy and produces working code, so... I leave this up to someone else.

[auscompgeek]

Looking at #7143, I think I can come up with a correct fix. Give me a few minutes.

[richvdh]

fixed by #7155