A.T.O.M Docs Ankara Telecom Optimization Model
Local data tooling

Dataset Pack Studio

Inspect, repair, reproject, package, validate, install, and safely switch arbitrary-region A.T.O.M dataset packs.

View Markdown source System architecture

Dataset Pack Studio builds portable, local A.T.O.M dataset packs for arbitrary regions. It inspects source coverage before build, repairs vector geometry, reprojects vectors to EPSG:4326, crops to an optional planning extent, writes deterministic GeoJSON, hashes every emitted layer, and produces a schema-v2 manifest with provenance and QA evidence.

It does not download data from a browser or upload source data to a service. Input processing and runtime switching remain local.

Environment

Install the hash-locked pipeline environment:

python3 -m venv data-pipeline/.venv
data-pipeline/.venv/bin/pip install --require-hashes -r data-pipeline/Requirements.txt

The Studio directly uses GeoPandas, Shapely, and PyProj. Supported vector inputs are formats readable by GeoPandas, including GeoJSON and GeoPackage. Terrain is copied as a binary layer and must have truthful CRS and unit metadata.

Inspect Sources Before Building

data-pipeline/.venv/bin/python data-pipeline/pack_studio.py inspect \
  --towers /data/izmir/cells.gpkg \
  --buildings /data/izmir/buildings.gpkg \
  --bounds 26.80,38.30,27.35,38.65 \
  --json

The preview reports source and output CRS, input/invalid/repaired/dropped/output geometry counts, per-field missing counts, output bounds, and requested-area coverage. Use --towers-crs or another --<layer>-crs override only when a source has no embedded CRS and the supplied value is authoritative.

Build A Schema-v2 Pack

data-pipeline/.venv/bin/python data-pipeline/pack_studio.py build \
  --id izmir-planning-2026 \
  --name "Izmir Planning Dataset" \
  --version 2026.07 \
  --output /datasets/izmir-planning-2026 \
  --source "Municipal building inventory, 2026-06" \
  --source "Operator-authorized planning export, 2026-07" \
  --license "Municipal Open Data License" \
  --license "Internal planning use only" \
  --confidence "Planning-grade cells; heights are partially surveyed." \
  --bounds 26.80,38.30,27.35,38.65 \
  --towers /data/izmir/cells.gpkg \
  --buildings /data/izmir/buildings.gpkg \
  --terrain /data/izmir/elevation.tif \
  --terrain-crs EPSG:32635 \
  --terrain-units m \
  --clutter /data/izmir/clutter.gpkg \
  --building-heights /data/izmir/heights.gpkg \
  --materials /data/izmir/materials.gpkg

The output directory must be absent or empty. The Studio stages the pack beside the destination and publishes it only after all layers and the manifest are complete. Required tower/building layers are normalized to towers.geojson and buildings.geojson; optional layers use stable filenames. Every referenced file receives a SHA-256 entry.

Manifest Schema v2

Schema v2 retains the required identity, EPSG:4326 bounds, sources, licenses, confidence note, files, and hashes from v1, and adds:

  • Per-layer kind, format, CRS, optionality, source, license, confidence, and optional units.
  • Geometry repair and output counts for each vector layer.
  • Missing-field counts and requested/data coverage bounds.
  • Optional terrain, clutter, building_heights, and materials files.

The engine loads and validates optional-layer metadata. A supported north-up EPSG:4326 COG/GeoTIFF terrain layer is sampled by the 2.5D point-to-point path profiler. Fast sectors, interference, optimization, batch runs, and analytical surfaces remain terrain-independent. Clutter and separate height/material sidecar layers are not automatically joined; user-selected path-profile sensitivity inputs must not be interpreted as surveyed layer values.

Validate And Install

Validate the finished pack with the same loader used by the server:

cd backend-go
go run ./cmd/validate-dataset /datasets/izmir-planning-2026

Set ATOM_DATASET_DIR to the initial pack. To enable the Data tool's installed-pack list and safe switching, set ATOM_DATASETS_ROOT to a directory that contains packs as immediate child directories:

/datasets/
├── ankara-open-planning/manifest.json
└── izmir-planning-2026/manifest.json

The runtime never accepts a filesystem path from the switch API. POST /api/datasets/switch accepts an installed manifest ID, resolves only entries beneath ATOM_DATASETS_ROOT, rejects duplicate IDs and symlink escapes, validates the candidate's geometry and hashes, then swaps the immutable in-memory pack. A failed load leaves the previous pack active. Configure DATASET_ADMIN_API_KEY at an origin gateway when dataset activation must be restricted.

Required Review

Before using a pack for a planning decision, review source authority, redistribution terms, missing RF fields, geometry repairs/drops, requested coverage ratio, coordinate reference systems, optional raster units, and whether confidence statements match the actual collection method. A successful hash or geometry validation proves integrity and structural usability, not operator accuracy.