Running Spatial Tests Without mod_spatialite

Split the suite by what it actually needs. Registry structure, geometry encoding, validity and the whole conformance band run on plain sqlite3 plus Shapely; only spatial SQL functions need the extension. Mark that subset with a requires_spatialite marker, skip it with a reason that names the missing capability, and report the skipped count in the build summary so the gap stays visible.

This page belongs to the Testing & CI for Spatial Pipelines guide.

Why This Matters #

Plenty of environments cannot load the extension. A Python built with SQLITE_OMIT_LOAD_EXTENSION — which several managed platforms ship deliberately — has no enable_load_extension at all. macOS strips the library search path from protected processes. Locked-down build runners forbid dynamic loading as a sandbox-escape route.

The usual response is to skip the whole spatial suite on those runners, which is a large loss for a small cause: most spatial assertions do not involve a spatial function. Registry integrity is ordinary SQL. Geometry decoding is Shapely. Validity is Shapely. Conformance is ordinary SQL. What genuinely needs the extension is ST_ functions evaluated inside SQLite, which is a minority of a well-structured suite.

Prerequisites #

• Python 3.9+ with pytest and shapely 2.0+
• A suite already split into the bands described in the parent guide
• A way to build fixtures that does not itself require the extension
• Agreement that a skipped test is a gap, not a pass

Primary Method #

# conftest.py — detect the capability once, and expose it as a marker
import sqlite3
import pytest


def _spatialite_available() -> tuple[bool, str]:
    if not hasattr(sqlite3.Connection, "enable_load_extension"):
        return False, "this Python was built with SQLITE_OMIT_LOAD_EXTENSION"
    conn = sqlite3.connect(":memory:")
    try:
        conn.enable_load_extension(True)
        conn.load_extension("mod_spatialite")
        conn.execute("SELECT spatialite_version()").fetchone()
        return True, ""
    except Exception as exc:
        return False, f"mod_spatialite did not load: {exc}"
    finally:
        conn.close()


HAS_SPATIALITE, SPATIALITE_REASON = _spatialite_available()


def pytest_configure(config):
    config.addinivalue_line(
        "markers", "requires_spatialite: needs ST_ functions inside SQLite"
    )


def pytest_collection_modifyitems(config, items):
    if HAS_SPATIALITE:
        return
    skip = pytest.mark.skip(reason=SPATIALITE_REASON)
    for item in items:
        if "requires_spatialite" in item.keywords:
            item.add_marker(skip)

Detecting once at collection, rather than per test, means the skip reason is uniform and the expensive load attempt happens a single time. Carrying the real exception text into the reason is what turns “24 skipped” into something someone can act on.

Step-by-Step Walkthrough #

1. Move assertions out of SQL where the Python equivalent is as good #

Many tests use a spatial function out of habit rather than necessity. A validity check written as SELECT count(*) WHERE ST_IsValid(geom) = 0 needs the extension; the same check written in Shapely does not, and runs everywhere.

# Needs the extension
@pytest.mark.requires_spatialite
def test_no_invalid_geometries_sql(conn):
    n = conn.execute(
        "SELECT count(*) FROM parcels WHERE ST_IsValid(geom) = 0"
    ).fetchone()[0]
    assert n == 0


# Runs anywhere, and asserts the same thing
def test_no_invalid_geometries(parcels_master):
    from shapely import from_wkb

    conn = sqlite3.connect(f"file:{parcels_master}?mode=ro", uri=True)
    bad = [
        fid for fid, blob in conn.execute(
            "SELECT rowid, geom FROM parcels WHERE geom IS NOT NULL"
        )
        if not from_wkb(gpb_to_wkb(blob)).is_valid
    ]
    assert not bad, f"invalid geometries at {bad}"

Both call GEOS; only one calls it through SQLite. Preferring the second wherever the assertion is about the data rather than about the SQL is what shrinks the extension-dependent set.

2. Keep the marker for tests that genuinely exercise SQL #

Some tests are specifically about the SQL path — that the query uses the index, that a search_frame subquery returns what it should, that ST_Transform behaves. Those belong behind the marker, because rewriting them in Python would test something else.

@pytest.mark.requires_spatialite
def test_search_frame_returns_the_window(conn):
    rows = conn.execute("""
        SELECT ROWID FROM SpatialIndex
        WHERE f_table_name = 'parcels'
          AND search_frame = BuildMbr(0, 0, 15, 15, 27700)
    """).fetchall()
    assert len(rows) == 2

3. Build fixtures without the extension #

A fixture built through Fiona or pyogrio needs GDAL, not mod_spatialite — GDAL has its own GeoPackage driver and does not use the SQLite extension. That means the fixture chain from How to Build a Test GeoPackage Fixture in pytest works unchanged on a runner where the extension is unavailable.

Where even GDAL is absent, a container can be built with sqlite3 alone by writing the registry rows and GeoPackage Binary blobs by hand, following the encoding in How to Serialize MultiPolygon Geometries to WKB in Python. It is more code, and it removes the last dependency.

4. Make the gap visible in the summary #

A skipped test that nobody reads becomes permanent. Report the count and the reason at the end of the run, so a runner silently losing the extension is noticed.

# conftest.py
def pytest_terminal_summary(terminalreporter, exitstatus, config):
    if HAS_SPATIALITE:
        return
    skipped = len(terminalreporter.stats.get("skipped", []))
    terminalreporter.write_sep(
        "=", f"{skipped} test(s) skipped — {SPATIALITE_REASON}", yellow=True
    )

5. Run the full suite somewhere #

The split makes a restricted runner useful; it does not make it sufficient. Arrange for at least one environment — a nightly job, a release build, a container image — where the extension loads and the marked tests actually run. Otherwise the marked set rots, and the first time it runs after six months it fails for reasons nobody remembers.

# One job runs everything; the fast job runs what it can
jobs:
  fast:
    runs-on: ubuntu-latest
    steps:
      - run: pytest -q                      # marked tests skip here

  full:
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.8.4
    steps:
      - run: apt-get update && apt-get install -y libsqlite3-mod-spatialite
      - run: pytest -q --strict-markers -W error

Verification #

The detection itself needs testing, because a false positive silently skips tests on a machine that could run them.

def test_detection_matches_reality():
    """If the extension loads here, HAS_SPATIALITE must say so."""
    conn = sqlite3.connect(":memory:")
    try:
        conn.enable_load_extension(True)
        conn.load_extension("mod_spatialite")
        loaded = True
    except Exception:
        loaded = False
    finally:
        conn.close()

    assert loaded == HAS_SPATIALITE, (
        f"detection says {HAS_SPATIALITE} but loading {'worked' if loaded else 'failed'}"
    )

And the full job should refuse to pass if the marked tests skipped there:

def test_marked_tests_must_run_in_the_full_job(request):
    """Guarded by an env var the full job sets."""
    import os

    if os.environ.get("REQUIRE_SPATIALITE") == "1":
        assert HAS_SPATIALITE, (
            f"the full job requires the extension: {SPATIALITE_REASON}"
        )

Without that assertion the full job degrades to the fast one the day the image changes, and the build stays green throughout.

Alternative Approaches or Edge Cases #

pysqlite3-binary. This package bundles its own SQLite with extension loading enabled, and swapping import sqlite3 for import pysqlite3 as sqlite3 is often enough to make the extension load on a platform whose system Python cannot. It changes which SQLite you are testing against, which is a real trade-off, but for a build runner it is frequently the right one.

GDAL instead of the extension. Where a test needs a spatial operation but not specifically the SQL path, GDAL’s own geometry functions provide it without the extension — and where GDAL is the pipeline’s real interface, testing through it is more representative anyway.

Testing that the graceful path works. If the production code has a fallback for a missing extension, that fallback needs a test — and the only environment where it runs naturally is the restricted one. A requires_no_spatialite marker, the mirror of the first, gives it somewhere to live.

Troubleshooting #

Tests skip locally where the extension is definitely installed #

Cause: The library is present but not on the loader’s search path, or the interpreter embeds a different SQLite. Fix: Print SPATIALITE_REASON — it carries the real exception, and the two cases produce quite different messages. Resolving mod_spatialite Load Errors Across Platforms covers both.

The full job passes with everything skipped #

Cause: The extension is not installed in that image and nothing asserts it should be. Fix: Add the REQUIRE_SPATIALITE assertion above. This is the failure the whole arrangement is most vulnerable to.

A test uses the extension without being marked #

Cause: The marker was forgotten, so it errors rather than skipping on a restricted runner. Fix: Run the suite once in an environment without the extension and mark everything that errors; adding --strict-markers prevents the reverse problem of a mistyped marker silently doing nothing.

Frequently Asked Questions #

Related #

• Testing & CI for Spatial Pipelines — parent guide: the four bands, and which need what
• Resolving mod_spatialite Load Errors Across Platforms — fixing the load rather than working around it
• How to Build a Test GeoPackage Fixture in pytest — fixtures that build without the extension
• Asserting OGC Compliance in Continuous Integration — a whole band that never needed it