Engines

Multi-engine DataFrame abstraction: pandas, DuckDB, Spark+Sedona, PostGIS. Same analysis at different scales.

Engine abstractions — engine-agnostic DataFrame operations.

Holds the DataFrameEngine ABC and its four concrete implementations (Pandas, DuckDB, Spark, PostGIS).

Submodules load on first attribute access via PEP 562 __getattr__.

class siege_utilities.engines.Engine

Bases: Enum

Supported DataFrame engine back-ends.

PANDAS = 'pandas'

DUCKDB = 'duckdb'

SPARK = 'spark'

POSTGIS = 'postgis'

class siege_utilities.engines.DataFrameEngine

Bases: ABC

Abstract base class for engine-agnostic DataFrame operations.

apportion(source_polys, target_polys, weight_col, *, source_geom='geometry', target_geom='geometry')

Apportion a value from source polygons to target polygons by overlap.

Distributes weight_col from source to target proportional to the fraction of source area that overlaps each target. Example: apportion tract population to congressional districts.

Default: intersect, compute area ratios via GeoPandas, aggregate. Engines may override for native area computation (Sedona ST_Area, PostGIS ST_Area, etc.).

Parameters:

• source_polys (Any)

• target_polys (Any)

• weight_col (str)

• source_geom (str)

• target_geom (str)

Return type:

Any

assign_boundaries(points, polygons, *, point_geom='geometry', polygon_geom='geometry', how='left')

Assign each point to its containing polygon(s).

Core DSTK replacement: given addresses (points) and boundaries (polygons), determine which boundary contains each address.

Default delegates to spatial_join() with predicate=’contains’ (reversed: polygon contains point). Engines may override for broadcast optimization or native spatial indexing.

Parameters:

• points (Any)

• polygons (Any)

• point_geom (str)

• polygon_geom (str)

• how (str)

Return type:

Any

abstractmethod buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

Parameters:

• df (Any)

• distance (float)

• geometry_col (str)

• crs (str | None)

Return type:

Any

dissolve(df, by, geometry_col='geometry', **agg_kwargs)

Dissolve/merge geometries grouped by by columns.

Default converts to GeoDataFrame and uses gdf.dissolve(). Engines may override for native performance.

Parameters:

• df (Any)

• by (str | List[str])

• geometry_col (str)

• agg_kwargs (Any)

Return type:

Any

abstractmethod distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

Parameters:

• df (Any)

• other (Any)

• geometry_col (str)

• other_geom (str)

Return type:

Any

abstractmethod filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert a GeoPandas GeoDataFrame to this engine’s native format.

The inverse of to_geodataframe(). Geometry is serialized as WKB-hex (DuckDB) or WKT (Spark/PostGIS) for engine-native spatial operations.

Subclasses should override this for engine-specific conversion. The default implementation returns the GeoDataFrame unchanged (suitable for PandasEngine).

Parameters:

• gdf (Any)

• geometry_col (str)

Return type:

Any

abstractmethod groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

index_points(df, lat_col, lon_col, *, grid=None, resolution=None, level=None, as_token=True)

Compute a grid (H3 or S2) cell ID for each row.

Default: round-trip through pandas. Engines that benefit from a native path (Spark via pandas UDF, DuckDB via SQL when the spatial extension is loaded) should override.

See siege_utilities.geo.grids.index_points() for the kwarg inference rules – pass resolution= for H3, level= for S2, or set grid= explicitly.

Returns the same engine-native frame type as df would have been augmented with – for the default, that’s a column appended to a pandas-converted copy. Engines that override return their native shape.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• as_token (bool)

Return type:

Any

index_polygon(geometry, *, grid=None, resolution=None, level=None, max_cells=None, min_level=None, max_level=None, refine=True)

Cover a polygon with grid cells.

Engine-agnostic by construction – the input is a single geometry, not a frame, so no engine-specific dispatch is required at this level. Provided here for API symmetry with index_points() and so consumer code can accept engine as a config knob and call engine.index_polygon(...) uniformly.

Parameters:

• geometry (Any)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• max_cells (int | None)

• min_level (int | None)

• max_level (int | None)

• refine (bool)

Return type:

Any

intersect_boundaries(poly_a, poly_b, *, a_geom='geometry', b_geom='geometry')

Find overlapping regions between two polygon boundary sets.

Returns all pairs of polygons that intersect, with both sets of attributes. Useful for county-to-district apportionment or boundary change analysis.

Default delegates to spatial_join() with predicate=’intersects’. Engines may override to compute intersection geometry and area.

Parameters:

• poly_a (Any)

• poly_b (Any)

• a_geom (str)

• b_geom (str)

Return type:

Any

abstractmethod join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

load_lines(path, *, geometry_col='geometry', format='auto', crs=None)

Load line/multiline geometries from any supported spatial format.

Same interface as load_polygons() – works for roads, rivers, transit routes, or any linear feature.

Parameters:

• path (str) – Path to the spatial file.

• geometry_col (str) – Name for the geometry column in the returned DataFrame. If the source data uses a different column name it will be renamed.

• format (str) – Spatial format hint. "auto" (default) infers from the file extension. Explicit values: "geojson", "shapefile", "gpkg", "geopackage", "parquet", "geoparquet".

• crs (str or None) – Target CRS. Defaults to EPSG:4326 via get_default_crs().

Return type:

Any

load_points(df, lat_col='lat', lon_col='lon', geometry_col='geometry')

Create point geometries from latitude/longitude columns.

Adds a geometry column with WKT POINT strings (or native geometry objects, depending on engine). Engines may override for native point construction.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• geometry_col (str)

Return type:

Any

load_polygons(path, *, geometry_col='geometry', format='auto', crs=None)

Load polygon geometries from any supported spatial format.

Handles GeoJSON, Shapefile, GeoParquet, GPKG, WKT CSV. Returns engine-native DataFrame with a geometry column.

Parameters:

• path (str) – Path to the spatial file.

• geometry_col (str) – Name for the geometry column in the returned DataFrame. If the source data uses a different column name it will be renamed.

• format (str) – Spatial format hint. "auto" (default) infers from the file extension. Explicit values: "geojson", "shapefile", "gpkg", "geopackage", "parquet", "geoparquet".

• crs (str or None) – Target CRS. Defaults to EPSG:4326 via get_default_crs().

Return type:

Any

multi_assign(points, boundary_layers, *, point_geom='geometry')

Assign points to multiple boundary layers simultaneously.

Calls assign_boundaries() for each layer and renames every polygon-side column to be {layer_name}_{column} so layers do not collide on shared schemas (TIGER layers share geoid / name columns – without prefixing, the second join silently overwrites the first layer’s columns).

Parameters:

• points (DataFrame) – Point DataFrame.

• boundary_layers (dict) – {layer_name: polygon_DataFrame}. Each polygon DataFrame must have a geometry column and identifying columns.

• point_geom (str)

Returns:

Points enriched with {layer_name}_* columns for every polygon-side column from each boundary layer. Bare polygon column names never appear in the output.

Return type:

DataFrame

abstract property name: str

Return the canonical engine name.

nearest(points, targets, *, k=1, max_distance=None, point_geom='geometry', target_geom='geometry')

Find the k nearest targets for each point.

Default: brute-force via GeoPandas sjoin_nearest (supports k=1 only). Engines should override for native spatial indexing (Sedona KNN, PostGIS ST_DWithin + ORDER BY distance, DuckDB spatial) and k>1.

Raises:

NotImplementedError – If k > 1 (base implementation only supports single-nearest).

Parameters:

• points (Any)

• targets (Any)

• k (int)

• max_distance (float | None)

• point_geom (str)

• target_geom (str)

Return type:

Any

point_in_polygon(points_df, polygons_df, *, points_geom='geometry', polygons_geom='geometry')

Find which polygon each point falls within.

Delegates to spatial_join() with predicate='within'.

Parameters:

• points_df (Any)

• polygons_df (Any)

• points_geom (str)

• polygons_geom (str)

Return type:

Any

abstractmethod query(sql, **kwargs)

Execute sql and return a DataFrame-like object.

Back-end specific keyword arguments (e.g. table for DuckDB, connection for PostGIS) are passed through via kwargs.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_csv(path, **kwargs)

Read a CSV file and return a DataFrame-like object.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_parquet(path, **kwargs)

Read a Parquet file and return a DataFrame-like object.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_spatial(path, *, crs=None, **kwargs)

Read a spatial file (GeoJSON, Shapefile, GeoParquet, GPKG).

Returns an engine-native spatial DataFrame. crs defaults to EPSG:4326 (via geo.crs.get_default_crs).

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

abstractmethod spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

Parameters:

• left (Any)

• right (Any)

• how (str)

• predicate (str)

• left_geom (str)

• right_geom (str)

Return type:

Any

abstractmethod to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

Parameters:

• df (Any)

• geometry_col (str)

• crs (str | None)

Return type:

Any

abstractmethod to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.PandasEngine

Bases: DataFrameEngine

Engine backed by pandas (and optionally GeoPandas).

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute SQL via pandas.read_sql.

Requires a connection keyword argument (a SQLAlchemy engine or DBAPI2 connection).

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file.

**kwargs: Forwarded to pandas.read_csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file.

**kwargs: Forwarded to pandas.read_parquet().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.DuckDBEngine

Bases: DataFrameEngine

Engine backed by DuckDB.

__init__(**kwargs)

Parameters:

kwargs (Any)

Return type:

None

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert GeoDataFrame to a pandas DataFrame with standard WKB geometry.

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute sql against the DuckDB connection.

If a table keyword is supplied together with a pandas DataFrame value, that DataFrame is first registered as a virtual table so it can be referenced in sql. Row and status results are eagerly materialized pandas DataFrames, independent of the connection lifetime and subsequent source mutations. Output columns whose DuckDB type is GEOMETRY (including DML RETURNING) are converted to standard WKB by column position before fetching; nested geometry values and other spatial types are not converted. Duplicate names retain their distinct values and receive DuckDB’s usual suffixes. to_pandas() returns this frame unchanged; to_geodataframe() decodes its WKB geometry. DML without RETURNING retains its affected-row Count; PREPARE returns its empty Success status without executing the prepared statement. Spatial is loaded only for geometry conversion or SQL requiring it.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV via DuckDB’s read_csv_auto.

Engine-specific kwargs (DuckDB’s read_csv_auto options like header, delim, columns) are not yet forwarded; pass the equivalent options inside a custom SQL via query() if you need to override auto-detection.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file via DuckDB’s read_parquet.

Engine-specific kwargs are not yet forwarded; use query() for DuckDB-specific options.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file via DuckDB’s ST_Read.

Geometry columns are identified by scalar type, not name, so a feature property named geom does not shadow the real geometry. The real columns are the attribute fields plus geometry_fields reported by ST_Read_Meta. The scan is bound with collision-proof positional aliases (DuckDB spatial 1.5.x rejects a SELECT * over duplicate source names instead of auto-renaming them); any leading driver-injected column such as OGC_FID – which 1.5.x emits, 1.4.x does not, and metadata does not list – is dropped so the output schema is identical across DuckDB versions. Names are then restored from metadata with pandas-style deduplication, so a clashing geom property stays addressable while the real geometry becomes geom_1.

**kwargs: Forwarded for interface compatibility (unused in the current DuckDB path).

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Decode the selected WKB column of a materialized query result.

Also accepts typed DuckDB relations (materialized via to_pandas()), frames containing hex WKB, WKT or Shapely geometry, and GeoDataFrames.

to_pandas(df)

Return a pandas frame, converting GEOMETRY columns in typed relations.

Frames returned by query() are already materialized and are returned unchanged. Relation columns typed exactly GEOMETRY become WKB; nested geometry values and other spatial types are not converted. Use to_geodataframe() to decode the WKB.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.SparkEngine

Bases: DataFrameEngine

Engine backed by PySpark.

Parameters:

spark (optional) – An existing SparkSession. When None the engine creates or retrieves the active session on first use.

__init__(spark=None, *, enable_sedona=False)

Parameters:

• spark (Any)

• enable_sedona (bool)

Return type:

None

assign_boundaries(points, polygons, *, point_geom='geometry', polygon_geom='geometry', how='left')

Point-in-polygon – Sedona ST_Contains with optional broadcast.

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert a GeoDataFrame to a Spark DataFrame with WKT geometry.

The base implementation returns the GeoDataFrame unchanged, which left boundary-provider conversion handing a GeoDataFrame into Spark-only calls (createOrReplaceTempView). Serialize geometry to WKT and build a real Spark DataFrame so the Spark spatial methods (which read ST_GeomFromText(geometry_col)) receive engine-native input.

Spark/Sedona carries no CRS metadata. Coordinates are serialized to WKT exactly as they are in gdf – no reprojection – so a read_spatial(crs=…) -> from_geodataframe -> to_geodataframe(crs=…) round-trip preserves the numbers unchanged. The caller is responsible for passing the same crs to to_geodataframe that the coordinates are actually in; normalizing to a default CRS here would corrupt data that entered in a projected CRS.

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

load_points(df, lat_col='lat', lon_col='lon', geometry_col='geometry')

Create point geometries – Spark-native WKT string construction.

load_polygons(path, *, geometry_col='geometry', format='auto', crs=None)

Load polygons – Spark-native parquet/json reader.

property name: str

Return the canonical engine name.

nearest(points, targets, *, k=1, max_distance=None, point_geom='geometry', target_geom='geometry')

K-nearest – Sedona ST_Distance with window function.

query(sql, **kwargs)

Execute sql via Spark SQL.

SparkEngine.query accepts no engine-specific kwargs; self._session.sql() takes no options beyond the SQL text. The **kwargs parameter is present for interface symmetry with DuckDBEngine (which accepts table / df) and PostGISEngine (which accepts geom_col). A non-empty kwargs is a caller error and is raised to surface the mistake rather than silently ignored.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file via Spark.

**kwargs: Forwarded to Spark’s DataFrameReader.csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file via Spark.

**kwargs: Forwarded to Spark’s DataFrameReader.parquet().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file via GeoPandas and convert to a Spark DataFrame.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.PostGISEngine

Bases: DataFrameEngine

Engine backed by PostGIS via SQLAlchemy + GeoPandas.

Parameters:

connection_string (str) – A SQLAlchemy connection string, e.g. "postgresql://user:pass@host:5432/dbname".

__init__(connection_string)

Parameters:

connection_string (str)

Return type:

None

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

index_points(df, lat_col, lon_col, *, grid=None, resolution=None, level=None, as_token=True)

PostGIS does not natively compute H3 / S2 cell IDs.

For S2-keyed warehouse joins, the recommended pattern is:

1. Compute cell IDs ONCE during ingest (e.g. via the PandasEngine or in your ETL job using s2_index_points).

2. Store as a BIGINT column on the row.

3. For region queries, generate (range_min, range_max) pairs via s2_cells_to_ranges() and filter with WHERE s2_cell BETWEEN range_min AND range_max.

Computing cell IDs at query time over PostGIS would defeat the whole point – the database can’t index a value it has to compute per-row. Hence this engine raises rather than silently degrade.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• as_token (bool)

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute sql against the PostGIS database.

Pass geom_col="<column>" to indicate a geometry column in the result; the query then returns a GeoDataFrame with that column parsed as geometry. Omit geom_col (or pass an empty string) for queries that return no geometry; the result is a plain DataFrame.

The Geo-vs-plain decision is made by the caller (via the presence of geom_col), not by exception-fallback. An earlier implementation tried gpd.read_postgis first and silently fell back to pd.read_sql on any exception – that pattern hid real PostGIS errors (a malformed SQL, a missing column) as “this must not have a geometry column.”

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file.

**kwargs: Forwarded to pandas.read_csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a parquet file. Returns a GeoDataFrame when the file carries GeoParquet metadata, a plain DataFrame otherwise.

The decision is by file inspection (pyarrow schema metadata), not by exception-fallback. An earlier implementation tried gpd.read_parquet first and silently fell back to pd.read_parquet on any exception – that pattern hid real gpd errors (a corrupt geoparquet file, an out-of-date geopandas) as “this must not be a geoparquet file.” The current path checks the parquet metadata directly so the decision is explicit.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file or execute a spatial SQL query.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

siege_utilities.engines.get_engine(name, **kwargs)

Return an engine instance for the requested back-end.

Parameters:

• name (str or Engine) – One of "pandas", "duckdb", "spark", "postgis" (or the corresponding Engine enum member).

• **kwargs – Passed to the engine constructor (e.g. connection_string for PostGIS, spark for Spark).

Return type:

DataFrameEngine

Submodules

Engine-agnostic DataFrame operations.

Provides a thin abstraction so the same analytical operations can run on DuckDB, Spark, or PostGIS instead of only pandas/GeoPandas.

Why one abstraction

The point is not that the back-ends are interchangeable in performance – they are not. The point is that consumer code should not branch on back-end. A reporting module asked to summarise voter rolls should not care whether the frame came from a 10-million-row Spark job or a 5-row pandas test fixture; it calls engine.groupby_agg(...) and is done.

This is load-bearing. The anti-pattern to avoid: if isinstance(df, pd.DataFrame): ... else: ... scattered through reporting/. If you find yourself reaching for it, the right fix is a new method on the DataFrameEngine interface – not a special case in the consumer.

Usage:

from siege_utilities.engines.dataframe_engine import get_engine, Engine

engine = get_engine(Engine.PANDAS)
df = engine.read_csv("data.csv")
result = engine.groupby_agg(df, ["state"], {"pop": "sum"})
pdf = engine.to_pandas(result)

Each back-end uses lazy imports so the module itself never requires DuckDB, PySpark, or SQLAlchemy/psycopg2 to be installed.

class siege_utilities.engines.dataframe_engine.Engine

Bases: Enum

Supported DataFrame engine back-ends.

PANDAS = 'pandas'

DUCKDB = 'duckdb'

SPARK = 'spark'

POSTGIS = 'postgis'

class siege_utilities.engines.dataframe_engine.DataFrameEngine

Bases: ABC

Abstract base class for engine-agnostic DataFrame operations.

apportion(source_polys, target_polys, weight_col, *, source_geom='geometry', target_geom='geometry')

Apportion a value from source polygons to target polygons by overlap.

Distributes weight_col from source to target proportional to the fraction of source area that overlaps each target. Example: apportion tract population to congressional districts.

Default: intersect, compute area ratios via GeoPandas, aggregate. Engines may override for native area computation (Sedona ST_Area, PostGIS ST_Area, etc.).

Parameters:

• source_polys (Any)

• target_polys (Any)

• weight_col (str)

• source_geom (str)

• target_geom (str)

Return type:

Any

assign_boundaries(points, polygons, *, point_geom='geometry', polygon_geom='geometry', how='left')

Assign each point to its containing polygon(s).

Core DSTK replacement: given addresses (points) and boundaries (polygons), determine which boundary contains each address.

Default delegates to spatial_join() with predicate=’contains’ (reversed: polygon contains point). Engines may override for broadcast optimization or native spatial indexing.

Parameters:

• points (Any)

• polygons (Any)

• point_geom (str)

• polygon_geom (str)

• how (str)

Return type:

Any

abstractmethod buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

Parameters:

• df (Any)

• distance (float)

• geometry_col (str)

• crs (str | None)

Return type:

Any

dissolve(df, by, geometry_col='geometry', **agg_kwargs)

Dissolve/merge geometries grouped by by columns.

Default converts to GeoDataFrame and uses gdf.dissolve(). Engines may override for native performance.

Parameters:

• df (Any)

• by (str | List[str])

• geometry_col (str)

• agg_kwargs (Any)

Return type:

Any

abstractmethod distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

Parameters:

• df (Any)

• other (Any)

• geometry_col (str)

• other_geom (str)

Return type:

Any

abstractmethod filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert a GeoPandas GeoDataFrame to this engine’s native format.

The inverse of to_geodataframe(). Geometry is serialized as WKB-hex (DuckDB) or WKT (Spark/PostGIS) for engine-native spatial operations.

Subclasses should override this for engine-specific conversion. The default implementation returns the GeoDataFrame unchanged (suitable for PandasEngine).

Parameters:

• gdf (Any)

• geometry_col (str)

Return type:

Any

abstractmethod groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

index_points(df, lat_col, lon_col, *, grid=None, resolution=None, level=None, as_token=True)

Compute a grid (H3 or S2) cell ID for each row.

Default: round-trip through pandas. Engines that benefit from a native path (Spark via pandas UDF, DuckDB via SQL when the spatial extension is loaded) should override.

See siege_utilities.geo.grids.index_points() for the kwarg inference rules – pass resolution= for H3, level= for S2, or set grid= explicitly.

Returns the same engine-native frame type as df would have been augmented with – for the default, that’s a column appended to a pandas-converted copy. Engines that override return their native shape.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• as_token (bool)

Return type:

Any

index_polygon(geometry, *, grid=None, resolution=None, level=None, max_cells=None, min_level=None, max_level=None, refine=True)

Cover a polygon with grid cells.

Engine-agnostic by construction – the input is a single geometry, not a frame, so no engine-specific dispatch is required at this level. Provided here for API symmetry with index_points() and so consumer code can accept engine as a config knob and call engine.index_polygon(...) uniformly.

Parameters:

• geometry (Any)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• max_cells (int | None)

• min_level (int | None)

• max_level (int | None)

• refine (bool)

Return type:

Any

intersect_boundaries(poly_a, poly_b, *, a_geom='geometry', b_geom='geometry')

Find overlapping regions between two polygon boundary sets.

Returns all pairs of polygons that intersect, with both sets of attributes. Useful for county-to-district apportionment or boundary change analysis.

Default delegates to spatial_join() with predicate=’intersects’. Engines may override to compute intersection geometry and area.

Parameters:

• poly_a (Any)

• poly_b (Any)

• a_geom (str)

• b_geom (str)

Return type:

Any

abstractmethod join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

load_lines(path, *, geometry_col='geometry', format='auto', crs=None)

Load line/multiline geometries from any supported spatial format.

Same interface as load_polygons() – works for roads, rivers, transit routes, or any linear feature.

Parameters:

• path (str) – Path to the spatial file.

• geometry_col (str) – Name for the geometry column in the returned DataFrame. If the source data uses a different column name it will be renamed.

• format (str) – Spatial format hint. "auto" (default) infers from the file extension. Explicit values: "geojson", "shapefile", "gpkg", "geopackage", "parquet", "geoparquet".

• crs (str or None) – Target CRS. Defaults to EPSG:4326 via get_default_crs().

Return type:

Any

load_points(df, lat_col='lat', lon_col='lon', geometry_col='geometry')

Create point geometries from latitude/longitude columns.

Adds a geometry column with WKT POINT strings (or native geometry objects, depending on engine). Engines may override for native point construction.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• geometry_col (str)

Return type:

Any

load_polygons(path, *, geometry_col='geometry', format='auto', crs=None)

Load polygon geometries from any supported spatial format.

Handles GeoJSON, Shapefile, GeoParquet, GPKG, WKT CSV. Returns engine-native DataFrame with a geometry column.

Parameters:

• path (str) – Path to the spatial file.

• geometry_col (str) – Name for the geometry column in the returned DataFrame. If the source data uses a different column name it will be renamed.

• format (str) – Spatial format hint. "auto" (default) infers from the file extension. Explicit values: "geojson", "shapefile", "gpkg", "geopackage", "parquet", "geoparquet".

• crs (str or None) – Target CRS. Defaults to EPSG:4326 via get_default_crs().

Return type:

Any

multi_assign(points, boundary_layers, *, point_geom='geometry')

Assign points to multiple boundary layers simultaneously.

Calls assign_boundaries() for each layer and renames every polygon-side column to be {layer_name}_{column} so layers do not collide on shared schemas (TIGER layers share geoid / name columns – without prefixing, the second join silently overwrites the first layer’s columns).

Parameters:

• points (DataFrame) – Point DataFrame.

• boundary_layers (dict) – {layer_name: polygon_DataFrame}. Each polygon DataFrame must have a geometry column and identifying columns.

• point_geom (str)

Returns:

Points enriched with {layer_name}_* columns for every polygon-side column from each boundary layer. Bare polygon column names never appear in the output.

Return type:

DataFrame

abstract property name: str

Return the canonical engine name.

nearest(points, targets, *, k=1, max_distance=None, point_geom='geometry', target_geom='geometry')

Find the k nearest targets for each point.

Default: brute-force via GeoPandas sjoin_nearest (supports k=1 only). Engines should override for native spatial indexing (Sedona KNN, PostGIS ST_DWithin + ORDER BY distance, DuckDB spatial) and k>1.

Raises:

NotImplementedError – If k > 1 (base implementation only supports single-nearest).

Parameters:

• points (Any)

• targets (Any)

• k (int)

• max_distance (float | None)

• point_geom (str)

• target_geom (str)

Return type:

Any

point_in_polygon(points_df, polygons_df, *, points_geom='geometry', polygons_geom='geometry')

Find which polygon each point falls within.

Delegates to spatial_join() with predicate='within'.

Parameters:

• points_df (Any)

• polygons_df (Any)

• points_geom (str)

• polygons_geom (str)

Return type:

Any

abstractmethod query(sql, **kwargs)

Execute sql and return a DataFrame-like object.

Back-end specific keyword arguments (e.g. table for DuckDB, connection for PostGIS) are passed through via kwargs.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_csv(path, **kwargs)

Read a CSV file and return a DataFrame-like object.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_parquet(path, **kwargs)

Read a Parquet file and return a DataFrame-like object.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

abstractmethod read_spatial(path, *, crs=None, **kwargs)

Read a spatial file (GeoJSON, Shapefile, GeoParquet, GPKG).

Returns an engine-native spatial DataFrame. crs defaults to EPSG:4326 (via geo.crs.get_default_crs).

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

abstractmethod spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

Parameters:

• left (Any)

• right (Any)

• how (str)

• predicate (str)

• left_geom (str)

• right_geom (str)

Return type:

Any

abstractmethod to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

Parameters:

• df (Any)

• geometry_col (str)

• crs (str | None)

Return type:

Any

abstractmethod to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.dataframe_engine.PandasEngine

Bases: DataFrameEngine

Engine backed by pandas (and optionally GeoPandas).

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute SQL via pandas.read_sql.

Requires a connection keyword argument (a SQLAlchemy engine or DBAPI2 connection).

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file.

**kwargs: Forwarded to pandas.read_csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file.

**kwargs: Forwarded to pandas.read_parquet().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.dataframe_engine.DuckDBEngine

Bases: DataFrameEngine

Engine backed by DuckDB.

__init__(**kwargs)

Parameters:

kwargs (Any)

Return type:

None

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert GeoDataFrame to a pandas DataFrame with standard WKB geometry.

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute sql against the DuckDB connection.

If a table keyword is supplied together with a pandas DataFrame value, that DataFrame is first registered as a virtual table so it can be referenced in sql. Row and status results are eagerly materialized pandas DataFrames, independent of the connection lifetime and subsequent source mutations. Output columns whose DuckDB type is GEOMETRY (including DML RETURNING) are converted to standard WKB by column position before fetching; nested geometry values and other spatial types are not converted. Duplicate names retain their distinct values and receive DuckDB’s usual suffixes. to_pandas() returns this frame unchanged; to_geodataframe() decodes its WKB geometry. DML without RETURNING retains its affected-row Count; PREPARE returns its empty Success status without executing the prepared statement. Spatial is loaded only for geometry conversion or SQL requiring it.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV via DuckDB’s read_csv_auto.

Engine-specific kwargs (DuckDB’s read_csv_auto options like header, delim, columns) are not yet forwarded; pass the equivalent options inside a custom SQL via query() if you need to override auto-detection.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file via DuckDB’s read_parquet.

Engine-specific kwargs are not yet forwarded; use query() for DuckDB-specific options.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file via DuckDB’s ST_Read.

Geometry columns are identified by scalar type, not name, so a feature property named geom does not shadow the real geometry. The real columns are the attribute fields plus geometry_fields reported by ST_Read_Meta. The scan is bound with collision-proof positional aliases (DuckDB spatial 1.5.x rejects a SELECT * over duplicate source names instead of auto-renaming them); any leading driver-injected column such as OGC_FID – which 1.5.x emits, 1.4.x does not, and metadata does not list – is dropped so the output schema is identical across DuckDB versions. Names are then restored from metadata with pandas-style deduplication, so a clashing geom property stays addressable while the real geometry becomes geom_1.

**kwargs: Forwarded for interface compatibility (unused in the current DuckDB path).

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Decode the selected WKB column of a materialized query result.

Also accepts typed DuckDB relations (materialized via to_pandas()), frames containing hex WKB, WKT or Shapely geometry, and GeoDataFrames.

to_pandas(df)

Return a pandas frame, converting GEOMETRY columns in typed relations.

Frames returned by query() are already materialized and are returned unchanged. Relation columns typed exactly GEOMETRY become WKB; nested geometry values and other spatial types are not converted. Use to_geodataframe() to decode the WKB.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.dataframe_engine.SparkEngine

Bases: DataFrameEngine

Engine backed by PySpark.

Parameters:

spark (optional) – An existing SparkSession. When None the engine creates or retrieves the active session on first use.

__init__(spark=None, *, enable_sedona=False)

Parameters:

• spark (Any)

• enable_sedona (bool)

Return type:

None

assign_boundaries(points, polygons, *, point_geom='geometry', polygon_geom='geometry', how='left')

Point-in-polygon – Sedona ST_Contains with optional broadcast.

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

from_geodataframe(gdf, geometry_col='geometry')

Convert a GeoDataFrame to a Spark DataFrame with WKT geometry.

The base implementation returns the GeoDataFrame unchanged, which left boundary-provider conversion handing a GeoDataFrame into Spark-only calls (createOrReplaceTempView). Serialize geometry to WKT and build a real Spark DataFrame so the Spark spatial methods (which read ST_GeomFromText(geometry_col)) receive engine-native input.

Spark/Sedona carries no CRS metadata. Coordinates are serialized to WKT exactly as they are in gdf – no reprojection – so a read_spatial(crs=…) -> from_geodataframe -> to_geodataframe(crs=…) round-trip preserves the numbers unchanged. The caller is responsible for passing the same crs to to_geodataframe that the coordinates are actually in; normalizing to a default CRS here would corrupt data that entered in a projected CRS.

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

load_points(df, lat_col='lat', lon_col='lon', geometry_col='geometry')

Create point geometries – Spark-native WKT string construction.

load_polygons(path, *, geometry_col='geometry', format='auto', crs=None)

Load polygons – Spark-native parquet/json reader.

property name: str

Return the canonical engine name.

nearest(points, targets, *, k=1, max_distance=None, point_geom='geometry', target_geom='geometry')

K-nearest – Sedona ST_Distance with window function.

query(sql, **kwargs)

Execute sql via Spark SQL.

SparkEngine.query accepts no engine-specific kwargs; self._session.sql() takes no options beyond the SQL text. The **kwargs parameter is present for interface symmetry with DuckDBEngine (which accepts table / df) and PostGISEngine (which accepts geom_col). A non-empty kwargs is a caller error and is raised to surface the mistake rather than silently ignored.

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file via Spark.

**kwargs: Forwarded to Spark’s DataFrameReader.csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a Parquet file via Spark.

**kwargs: Forwarded to Spark’s DataFrameReader.parquet().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file via GeoPandas and convert to a Spark DataFrame.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

class siege_utilities.engines.dataframe_engine.PostGISEngine

Bases: DataFrameEngine

Engine backed by PostGIS via SQLAlchemy + GeoPandas.

Parameters:

connection_string (str) – A SQLAlchemy connection string, e.g. "postgresql://user:pass@host:5432/dbname".

__init__(connection_string)

Parameters:

connection_string (str)

Return type:

None

buffer(df, distance, geometry_col='geometry', *, crs=None)

Buffer geometries by distance (in CRS units).

Returns a new DataFrame with the geometry column replaced by buffered geometries.

distance(df, other, geometry_col='geometry', other_geom='geometry')

Compute distances between geometries.

other may be a DataFrame (row-aligned pairwise) or a single geometry (shapely object / WKT string).

filter(df, condition)

Return rows of df that satisfy condition.

The type of condition is engine-specific (boolean Series for pandas, SQL expression string for DuckDB/Spark, etc.).

Parameters:

• df (Any)

• condition (Any)

Return type:

Any

groupby_agg(df, group_cols, agg_dict)

Group df by group_cols and aggregate according to agg_dict.

agg_dict maps column names to aggregation function names ("sum", "mean", "count", "min", "max").

Parameters:

• df (Any)

• group_cols (Sequence[str])

• agg_dict (Dict[str, str])

Return type:

Any

index_points(df, lat_col, lon_col, *, grid=None, resolution=None, level=None, as_token=True)

PostGIS does not natively compute H3 / S2 cell IDs.

For S2-keyed warehouse joins, the recommended pattern is:

1. Compute cell IDs ONCE during ingest (e.g. via the PandasEngine or in your ETL job using s2_index_points).

2. Store as a BIGINT column on the row.

3. For region queries, generate (range_min, range_max) pairs via s2_cells_to_ranges() and filter with WHERE s2_cell BETWEEN range_min AND range_max.

Computing cell IDs at query time over PostGIS would defeat the whole point – the database can’t index a value it has to compute per-row. Hence this engine raises rather than silently degrade.

Parameters:

• df (Any)

• lat_col (str)

• lon_col (str)

• grid (str | None)

• resolution (int | None)

• level (int | None)

• as_token (bool)

Return type:

Any

join(left, right, on, how='inner')

Join left and right on column(s) on using join type how.

Parameters:

• left (Any)

• right (Any)

• on (str | List[str])

• how (str)

Return type:

Any

property name: str

Return the canonical engine name.

query(sql, **kwargs)

Execute sql against the PostGIS database.

Pass geom_col="<column>" to indicate a geometry column in the result; the query then returns a GeoDataFrame with that column parsed as geometry. Omit geom_col (or pass an empty string) for queries that return no geometry; the result is a plain DataFrame.

The Geo-vs-plain decision is made by the caller (via the presence of geom_col), not by exception-fallback. An earlier implementation tried gpd.read_postgis first and silently fell back to pd.read_sql on any exception – that pattern hid real PostGIS errors (a malformed SQL, a missing column) as “this must not have a geometry column.”

Parameters:

• sql (str)

• kwargs (Any)

Return type:

Any

read_csv(path, **kwargs)

Read a CSV file.

**kwargs: Forwarded to pandas.read_csv().

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_parquet(path, **kwargs)

Read a parquet file. Returns a GeoDataFrame when the file carries GeoParquet metadata, a plain DataFrame otherwise.

The decision is by file inspection (pyarrow schema metadata), not by exception-fallback. An earlier implementation tried gpd.read_parquet first and silently fell back to pd.read_parquet on any exception – that pattern hid real gpd errors (a corrupt geoparquet file, an out-of-date geopandas) as “this must not be a geoparquet file.” The current path checks the parquet metadata directly so the decision is explicit.

Parameters:

• path (str)

• kwargs (Any)

Return type:

Any

read_spatial(path, *, crs=None, **kwargs)

Read a spatial file or execute a spatial SQL query.

**kwargs: Forwarded to geopandas.read_file().

Parameters:

• path (str)

• crs (str | None)

• kwargs (Any)

Return type:

Any

spatial_join(left, right, how='inner', predicate='intersects', *, left_geom='geometry', right_geom='geometry')

Spatial join using a geometric predicate.

predicate is one of: intersects, contains, within, touches, crosses, overlaps.

to_geodataframe(df, geometry_col='geometry', *, crs=None)

Convert df to a GeoPandas GeoDataFrame.

Always returns a geopandas.GeoDataFrame regardless of engine.

to_pandas(df)

Convert df to a pandas DataFrame.

Parameters:

df (Any)

Return type:

Any

siege_utilities.engines.dataframe_engine.get_engine(name, **kwargs)

Return an engine instance for the requested back-end.

Parameters:

• name (str or Engine) – One of "pandas", "duckdb", "spark", "postgis" (or the corresponding Engine enum member).

• **kwargs – Passed to the engine constructor (e.g. connection_string for PostGIS, spark for Spark).

Return type:

DataFrameEngine