Testing & CI for Spatial Pipelines

Spatial pipelines resist testing for a specific reason: the interesting behaviour lives in a C library stack that the test process does not control. A geometry predicate is GEOS, a reprojection is PROJ, a layer write is GDAL, and each of the three can change answer between one machine and another without anything in your code changing at all. Tests written as though the stack were deterministic pass locally and fail in the build, or worse, pass in both while the pipeline quietly does the wrong thing.

This guide is part of the Python Integration & Database Workflows section. It covers how to build fixtures that are actually reproducible, what to assert about a container rather than about a library, and how to pin an environment so a green build means something.

Prerequisites #

• Python 3.9+ with pytest 7+ — the fixture patterns here use tmp_path and session-scoped fixtures
• A container-building path you control — either GDAL/OGR or raw sqlite3, as covered in GDAL vs sqlite3 Module for GeoPackage Writes
• A CI runner that can install system libraries, or a base image that already carries GDAL, PROJ and GEOS
• Version pinning in place for the Python packages and the libraries beneath them
• Familiarity with OGC conformance checks from How to Validate GeoPackage OGC Compliance

Concept & Specification Reference #

Spatial tests fall into four bands, and mixing them is the usual source of a suite that is both slow and uninformative.

The first band is fast, deterministic and entirely yours to get right. The second is fast and objective — conformance is a property of the file, not an opinion. The third is where domain judgement lives and where most of the value is. The fourth is not really a test of your code at all; it is a test of the machine, and it belongs at the start of the run so that a broken environment fails in seconds rather than producing forty confusing assertion errors.

Step-by-Step Implementation #

1. Assert the environment before anything else #

The single highest-value test in a spatial suite is the one that refuses to run the rest on the wrong stack.

# tests/test_environment.py — runs first, fails fast
import sqlite3
import pytest
from packaging.version import Version

MIN = {"GDAL": "3.4.0", "PROJ": "8.0.0", "GEOS": "3.9.0"}


def test_library_versions():
    from osgeo import gdal
    import pyproj
    import shapely

    assert Version(gdal.__version__) >= Version(MIN["GDAL"])
    assert Version(pyproj.proj_version_str) >= Version(MIN["PROJ"])
    assert Version(shapely.geos_version_string.split("-")[0]) >= Version(MIN["GEOS"])


def test_proj_data_resolves():
    """A missing proj.db makes every transformation return silently wrong values."""
    from pyproj import Transformer

    t = Transformer.from_crs("EPSG:4326", "EPSG:3857", always_xy=True)
    x, y = t.transform(0.0, 0.0)
    assert abs(x) < 1e-6 and abs(y) < 1e-6, "PROJ data is not resolving"


def test_gpkg_driver_present():
    from osgeo import ogr

    assert ogr.GetDriverByName("GPKG") is not None


def test_spatialite_loads():
    conn = sqlite3.connect(":memory:")
    conn.enable_load_extension(True)
    conn.load_extension("mod_spatialite")
    assert conn.execute("SELECT spatialite_version()").fetchone()[0]

Each of these fails in milliseconds and each names exactly one thing. That is the point: an environment test is worthless if its failure message requires investigation.

2. Build fixtures that are reproducible byte for byte #

A test container built by GDAL is not byte-identical between runs — timestamps, page allocation and the reference-system catalogue all vary. Asserting on file hashes is therefore a trap. Assert on content instead, and build the fixture from an explicit, ordered specification so the content is deterministic even when the bytes are not.

# tests/conftest.py — a deterministic feature fixture
import pytest
from pathlib import Path
import fiona
from fiona.crs import CRS

SCHEMA = {
    "geometry": "MultiPolygon",
    "properties": {"parcel_ref": "str:32", "area_m2": "float", "surveyed": "int"},
}

# Fixed, hand-written features — no randomness, no generated coordinates
FEATURES = [
    {
        "geometry": {"type": "MultiPolygon",
                     "coordinates": [[[(0, 0), (0, 10), (10, 10), (10, 0), (0, 0)]]]},
        "properties": {"parcel_ref": "P-001", "area_m2": 100.0, "surveyed": 1},
    },
    {
        "geometry": {"type": "MultiPolygon",
                     "coordinates": [[[(20, 0), (20, 5), (25, 5), (25, 0), (20, 0)]]]},
        "properties": {"parcel_ref": "P-002", "area_m2": 25.0, "surveyed": 0},
    },
]


@pytest.fixture
def parcels_gpkg(tmp_path: Path) -> Path:
    """A minimal, fully specified GeoPackage — same content on every run."""
    path = tmp_path / "parcels.gpkg"
    with fiona.open(
        str(path), "w", driver="GPKG", layer="parcels",
        schema=SCHEMA, crs=CRS.from_epsg(4326),
    ) as dst:
        dst.writerecords(FEATURES)
    return path

Two features are enough for most tests, and two is deliberate: a fixture with two thousand generated features is slower, harder to reason about, and no more likely to catch a bug than one whose contents you can hold in your head. Where volume genuinely matters — an index-behaviour test, a memory-bound streaming test — build that fixture separately and mark it slow.

3. Assert structure, not implementation #

A structural test asks whether the container conforms. It should read like the specification, not like the code that produced it.

# tests/test_structure.py — assertions the OGC specification justifies
import sqlite3


def registry(path, sql, *params):
    conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
    try:
        return conn.execute(sql, params).fetchall()
    finally:
        conn.close()


def test_required_tables_present(parcels_gpkg):
    names = {r[0] for r in registry(
        parcels_gpkg, "SELECT name FROM sqlite_master WHERE type='table'"
    )}
    for required in ("gpkg_contents", "gpkg_geometry_columns", "gpkg_spatial_ref_sys"):
        assert required in names, f"missing required table {required}"


def test_registries_agree(parcels_gpkg):
    mismatches = registry(parcels_gpkg, """
        SELECT c.table_name FROM gpkg_contents c
        JOIN gpkg_geometry_columns g ON g.table_name = c.table_name
        WHERE c.data_type = 'features' AND c.srs_id <> g.srs_id
    """)
    assert not mismatches, f"srs_id disagrees for {mismatches}"


def test_every_layer_is_registered(parcels_gpkg):
    declared = {r[0] for r in registry(
        parcels_gpkg, "SELECT table_name FROM gpkg_contents WHERE data_type='features'"
    )}
    assert declared, "no feature layers registered"
    real = {r[0] for r in registry(
        parcels_gpkg, "SELECT name FROM sqlite_master WHERE type='table'"
    )}
    assert declared <= real, f"registry names tables that do not exist: {declared - real}"

These tests do not know how the container was built, which is exactly what makes them useful: switching the pipeline from Fiona to raw sqlite3 should not change a single line of them.

4. Test semantics with tolerances, not equality #

Geometry comparisons that use equality fail for reasons that have nothing to do with correctness — a vertex reordered by normalisation, a coordinate a floating-point unit away, a type promoted from Polygon to MultiPolygon. Compare geometrically.

# tests/test_semantics.py — compare geometry the way geometry compares
from shapely import from_wkb
from shapely.testing import assert_geometries_equal


def test_reprojection_preserves_shape(parcels_gpkg, reprojected_gpkg):
    src = load_geometry(parcels_gpkg, "P-001")
    dst = load_geometry(reprojected_gpkg, "P-001")

    # Not: src == dst. Not: src.wkb == dst.wkb.
    assert src.geom_type == dst.geom_type
    assert len(src.geoms) == len(dst.geoms)
    # Relative area is preserved by a conformal projection to within tolerance
    assert dst.area == pytest.approx(expected_area_m2, rel=1e-6)


def test_repair_does_not_move_vertices(invalid_geom):
    from shapely import make_valid

    fixed = make_valid(invalid_geom)
    original_coords = set(invalid_geom.exterior.coords)
    fixed_coords = {c for g in fixed.geoms for c in g.exterior.coords}
    assert fixed_coords <= original_coords, "repair introduced a vertex"

5. Pin the stack, not just the wheels #

Pinning requirements.txt fixes the Python packages and says nothing about the libraries they bind to. On a runner that installs GDAL from the system package manager, the same pinned GDAL==3.8.4 wheel can sit on top of two different PROJ builds. Pin the image.

# Pin the whole stack, not just the Python layer
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4

RUN apt-get update && apt-get install -y --no-install-recommends \
        libsqlite3-mod-spatialite=5.0.1-3 \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.lock /tmp/
RUN pip install --no-deps -r /tmp/requirements.lock

# Assert at build time, so a base-image change fails the build rather than the tests
RUN python -c "import pyproj; assert pyproj.proj_version_str.startswith('9.')"

The final RUN is the important line. Base images are rebuilt, and a tag that pointed at one PROJ version last month can point at another today; asserting during the build turns that into a red build with an obvious cause rather than a fleet of confusing test failures.

Validation & Verification #

A spatial test suite should be able to answer three questions about itself. Does it fail when the environment is wrong? Does it fail when the container is malformed? Does it fail when the answer is wrong? A suite that only ever goes green has not been shown to do any of them.

The cheapest way to check is to break things deliberately, once, and confirm each band notices:

# Confirm each band actually fails when it should
PROJ_LIB=/nonexistent pytest tests/test_environment.py   # expect: PROJ test fails
python -c "import sqlite3; \
  c=sqlite3.connect('fixture.gpkg'); \
  c.execute('DROP TABLE gpkg_geometry_columns'); c.commit()"
pytest tests/test_structure.py                            # expect: structure test fails

This is the same principle the rest of this site applies to validation gates: an assertion that has never rejected anything has not been shown to assert anything.

Common Failure Modes & Fixes #

Tests pass locally and fail in CI with a geometry difference #

Diagnosis: Different GEOS versions. Predicate results and make_valid output both changed across GEOS 3.8, 3.9 and 3.11, so a test asserting an exact repaired geometry is really asserting a GEOS version. Fix: Pin GEOS in the image and assert the version in the environment band, then relax the geometry assertion to a tolerance-based comparison so a future upgrade is a decision rather than a surprise.

A test container is empty and every assertion passes #

Diagnosis: The fixture wrote zero features — commonly because the writer’s with block raised and the test only checked that the file exists. An empty layer satisfies most structural assertions trivially. Fix: Assert a feature count in the fixture itself, before returning the path. A fixture that can silently produce nothing is worse than no fixture.

The suite is slow because every test builds a container #

Diagnosis: Function-scoped fixtures rebuilding the same file dozens of times. Fix: Use a session-scoped fixture for read-only containers and copy it per test where a test needs to write. Copying a small GeoPackage is far cheaper than building one through GDAL.

CI passes but the deployed pipeline produces wrong coordinates #

Diagnosis: The build image and the runtime image have different PROJ grid data, so transformations take different routes — the failure described in Datum Transformations & Projection Accuracy. Fix: Test against the runtime image, not a separate test image, and assert the transformation route rather than only the endpoints.

mod_spatialite loads in tests and not in production #

Diagnosis: The test process and the service run different Python builds, and only one has extension loading compiled in. Fix: Include the extension-load check in the environment band and run that band against the production image as a smoke test, not only against the test image.

Performance Notes #

The slowest thing in a spatial test suite is almost never the geometry. It is process and driver startup: importing GDAL, registering drivers, opening proj.db, and creating containers. A suite that builds one container per test spends most of its wall clock in setup.

Three changes usually recover the bulk of it. Scope read-only fixtures to the session so the container is built once. Use in-memory SQLite for anything that does not specifically test file behaviour — extension loading, geometry functions and registry queries all work identically. And run the environment band first with -x, so a broken image costs seconds rather than a full suite.

Where a test genuinely needs volume, generate the features once and keep the container as a checked-in artefact rather than building it in the fixture. A committed fixture is a dependency to maintain, but for index-behaviour and streaming tests it converts minutes of build time per run into a file read.

Child Pages #

Pages in this section go deeper on individual testing tasks:

• How to Build a Test GeoPackage Fixture in pytest — deterministic fixtures and the scoping that keeps them fast
• Asserting OGC Compliance in Continuous Integration — turning the conformance checks into a build gate
• Pinning GDAL, PROJ and GEOS Versions in CI — pinning the stack under the wheels
• Property-Based Testing for Geometry Round-Trips — generating the inputs a hand-written fixture never contains
• How to Benchmark a Spatial Query Regression — catching a lost index before a user does
• Running Spatial Tests Without mod_spatialite — keeping a suite useful on a restricted runner

Frequently Asked Questions #

Related #

• Python Integration & Database Workflows — parent section: the full Python spatial SQLite stack
• End-to-End Spatial Automation Recipes — the pipeline shape these tests are written against
• How to Validate GeoPackage OGC Compliance — the conformance checks a structural test asserts
• Datum Transformations & Projection Accuracy — why a pinned image matters more than a pinned wheel