MAPPER

Multimodal Accessibility and Place Profiling Engine for Arbitrary Regions.
Open-data pipeline that turns OSM, GTFS, and census inputs into multimodal, H3-aligned accessibility fields.

Builds mode-specific travel-time graphs, harmonizes 130+ opportunity layers, runs network KDE to produce continuous accessibility fields, and exports them on a national H3 grid — ready for digital twins, GIS, and GeoAI workflows.

---

Why MAPPER

• Scalable and reproducible — tested at country scale (Germany), runs on a single workstation for city regions.
• Multimodal by default — walk, bike, drive, and walk+transit with real GTFS timetables; each mode produces its own output columns.
• Harmonized opportunity layers — OSM POI and workplace/census attributes in a single schema.
• H3-native output — every result lands on a uniform hexagonal grid for direct use with DT/GeoAI stacks.
• Python-free runtime pipeline — native C++ handles configuration, graph and source preparation, NKDE, H3 sampling, Parquet output, and manifests; a shell script orchestrates the stage binaries.

---

Installation

A clone builds and runs with no conda environment and no Python. The pipeline is a set of native binaries; everything they link is built into one self-contained prefix.

Requirements

• Linux (Ubuntu 22.04+)
• GCC ≥ 11, CMake ≥ 3.25 (Arrow 19 needs 3.25)
• curl, tar, make, git
• No sudo. Nothing is installed outside your home directory.

You do not need dev packages for zlib, sqlite, expat, PROJ, libzip or Arrow — the bootstrap builds them. That is deliberate: a machine without dev headers (for example one that gets its libraries from conda) is common, and it is what used to force conda onto the runtime path.

Clone and bootstrap

git clone https://github.com/TUMFTM/MAPPER.git
cd MAPPER
bash source/infra/bootstrap_cpp_worker.sh

That single command:

1. fetches the header-only deps (libosmium, protozero, h3) into source/deps/
2. builds the compiled deps into ~/.local/share/mapper/deps/ (override with NATIVE_DEPS_PREFIX) — zlib, bzip2, expat, SQLite, PROJ (with its own proj.db), libzip, and a static parquet-only Arrow
3. builds every library and CLI into source/cpp/.build/
4. smoke-tests the binaries and asserts they depend on no conda env

Expect ~20–30 minutes on a first run, most of it Arrow. Re-running is cheap: each step builds only what is missing.

To verify independently at any time:

bash source/infra/check_no_conda.sh

It runs ldd over every binary with conda deliberately off the library path — the only way to prove they resolve unaided — and fails if anything links back into an environment.

Optional Python visualization

Python is gone from the pipeline, not from the repository. The offline figure utilities under source/viz/ use pyarrow, h3, pandas, and plotting libraries. They run separately on a workstation and are never invoked by a pipeline run. To use them:

python3 -m venv .venv && . .venv/bin/activate
pip install -r source/viz/requirements.txt

No Conda environment is required for the build or runtime pipeline.

---

The mapper front end

One binary drives everything, and with no arguments it opens a terminal UI:

source/cpp/.build/bin/mapper           # interactive: pick a config, fetch inputs, run, watch
mapper run    --config <f.toml>        # same run, non-interactively
mapper fetch  osm --region europe/germany/saarland
mapper fetch  gtfs                     # explains what to obtain; does NOT download
mapper config --check <f.toml>         # validate before committing hours to a run
mapper config --explain                # every key, its type, its default
mapper --version                       # version, commit, build date, CPU baseline

The TUI lists the configs it finds, shows each one's region, modes, kernel and layer count, and tells you whether that config's inputs are actually on disk before you start. r runs, f fetches the OSM extract, v validates, q quits.

Nothing in it is a second implementation: it drives the same run_native.sh the command line does, and reads config metadata from mapper_config_cpp, so the UI cannot disagree with the pipeline or offer a setting the pipeline no longer has.

---

Quickstart — verify the installation

# Run the shipped synthetic fixture (examples/selftest_bremen.*) and compare a complete
# walk/bike/drive run with the checked-in reference summary.
source/cpp/.build/bin/mapper selftest

The self-test uses an included checksum-pinned synthetic network in Bremen's CRS, requires no network connection or GTFS feed, and removes its temporary output after a successful comparison. A normal run writes cells.parquet (one row per H3 cell, one column per mode × opportunity layer) plus run_manifest.json containing the resolved configuration, cache digests, per-stage timings, and peak RSS.

---

Configuration

Configs are TOML. Validate before running — unknown or mistyped keys are errors, which is the point:

source/cpp/.build/bin/mapper_config_cpp --config <config.toml> --emit-flat

The flat KEY = value form that command prints is an internal artifact: it is what the cache digests are computed over. run_native.sh accepts one, but writing it by hand skips validation, so a typo becomes a silently substituted default.

A minimal config needs only a region, modes and the opportunity layers:

[run]
region = "europe/germany/hessen"
modes  = ["walk", "drive", "walk_transit"]

[input]
gtfs = "data/gtfs_germany.zip"        # no pbf key: derived from the region slug

[sources]
poi_classes = ["supermarket", "school", "pharmacy"]

[kernel]
type     = "gaussian"
param    = 600
cutoff_s = 2700

[output]
h3_res = 10

Example configs live in examples/. examples/selftest_bremen.toml is the reference fixture mapper selftest uses; examples/hessen_full.toml is a full-size run with every mode and every opportunity class.

---

Input Data

OSM (always required)

Fetch the extract with bash source/fetch_pbf.sh <slug>. The region key in your config must be the exact Geofabrik path slug for your area — copy it from the URL structure on the Geofabrik downloads page. That slug is also what the extract's local path is derived from, so it is stated once and only once.

A Geofabrik extract is the smallest unit the pipeline works in; there is no sub-region clipping.

https://download.geofabrik.de/europe/germany/hamburg.html
                               ↑ slug: "europe/germany/hamburg"

https://download.geofabrik.de/europe/france.html
                               ↑ slug: "europe/france"

Extracts are cached under data/cache/input/ and reused on subsequent runs. A run is never allowed to download anything itself: a mistyped slug should not pull several GB, and a pipeline that reaches out to the network mid-run is not reproducible from local inputs.

GTFS (required for transit modes only)

Transit modes (transit, walk_transit, walk_transit_bus, walk_transit_rail) require a GTFS schedule file. OSM-only modes (walk, bike, drive) work without one.

GTFS is not fetched automatically and is not redistributed in this repo — feeds carry their own licences. Download one and point the config at it:

[input]
gtfs      = "data/gtfs_germany.zip"
# Optional: route-type subsets, for the bus-only and rail-only modes.
gtfs_bus  = "data/cache/input/gtfs_route_subsets/gtfs_germany_routes-3.zip"
gtfs_rail = "data/cache/input/gtfs_route_subsets/gtfs_germany_routes-0-1-2-5-6-7.zip"

The path is relative to the repo root, or absolute. One nationwide feed serves every region inside it — the pipeline clips the feed against the region's boundary ring itself, which is why transit build time barely depends on region size.

GTFS feeds are available from transit agencies directly, or from aggregators such as Mobility Database and transit.land.

---

Running

The mapper front end provides both a TUI and a non-interactive command. The underlying shell orchestrator is also available directly:

bash source/run_native.sh --config <config.toml> [--out-dir DIR] [--force]

Stages are skipped when their output already exists, so a re-run resumes; --force rebuilds everything. Useful knobs:

variable	meaning
PREP_JOBS	prep-stage concurrency (default 3). Peak RSS becomes roughly the sum of concurrent stages rather than the max — set 1 on a memory-tight host.
KEEP_CELLS_BIN	keep the streaming scratch file after the parquet is written (debugging the writer).
NATIVE_DEPS_PREFIX	where the compiled deps live (default ~/.local/share/mapper/deps).

Progress goes to stderr, including the heartbeat. When logging to a file, keep it:

bash source/run_native.sh --config c.toml 2>&1 | tee run.log

The individual stage binaries in source/cpp/.build/bin/ are usable on their own; each takes --help. run_native.sh is the orchestration, not the implementation.

---

Adding a New Region

cp examples/hessen_full.toml my_city.toml

Edit the region, the modes and the layers:

[run]
region = "europe/germany/berlin"     # Geofabrik slug, exactly
modes  = ["walk", "walk_transit"]

[sources]
poi_classes = ["hospital", "school", "supermarket"]

[kernel]
cutoff_s = 1800                      # 30 minutes

Then:

bash source/fetch_pbf.sh europe/germany/berlin
bash source/run_native.sh --config my_city.toml

Every key, with defaults and its flat-form name, is in docs/configuration.md; mapper_config_cpp --config <file> --emit-flat shows exactly what a config resolves to.

---

C++ Libraries

Everything is built by bash source/infra/bootstrap_cpp_worker.sh. To rebuild manually:

# Full rebuild (deps + all targets + smoke tests)
bash source/infra/bootstrap_cpp_worker.sh

# Individual targets
bash source/cpp/build.sh graph     # libgraph.so     — OSM/GTFS parser
bash source/cpp/build.sh nkde      # libroutingkit.so — RoutingKit NKDE + hex sampler
bash source/cpp/build.sh extended  # libgraph_ext.so  — preprocessing
bash source/cpp/build.sh pipeline  # run_pipeline_cpp — native orchestrator

Artifacts land in source/cpp/.build/ — lib/ for the shared libraries, bin/ for the binaries.

---

Documentation

File	Contents
docs/configuration.md	Full config key reference with defaults
docs/architecture.md	Native pipeline stages and component boundaries
docs/deployment.md	Detailed install, HPC, and container setup
docs/troubleshooting.md	Common issues and fixes
CHANGELOG.md	Release history and upgrade-impact summary

---

Citing

If you use MAPPER, please cite the software and the accompanying paper. Machine-readable metadata is in CITATION.cff (GitHub's "Cite this repository" button reads it).

• Software: Till Zacher, Louis Stille-Hönig, Markus Lienkamp. MAPPER: Multimodal Accessibility and Place Profiling Engine for Arbitrary Regions, version 2.0.0, 2026. https://github.com/TUMFTM/MAPPER
• Paper: Till Zacher, Louis Stille-Hönig, Markus Lienkamp. National-Scale Multimodal Accessibility Fields from Open Geospatial Data. Preprint, 2026.

---

License

Apache License 2.0 — see LICENSE and NOTICE. Input-data licences (OSM, GTFS) are summarised in DATA_LICENSES.md; bundled third-party components in THIRD_PARTY_NOTICES.md.