Require HTTPS consistently for catalog sources

.github/security-audit-requirements.txt

@@ -315,7 +315,9 @@ cryptography==50.0.2 \
     --hash=sha256:f9f6143a8c75945eb960d9eb98905a441394abfa24afaae239d514ffb2586480 \
     --hash=sha256:fa8f5efb344d6908a1ce62f4a24e2e5780f825d6f53f5f50ec5ffacac72936cb \
     --hash=sha256:fdd28f912fccfec1846a94e2e1e8f9b0012f557f0c46fe4f3eb0d7a87afcf90b
-    # via pyjwt
+    # via
+    #   specify-cli (pyproject.toml)
+    #   pyjwt
 execnet==2.1.2 \
     --hash=sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd \
     --hash=sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec

design/catalogs.md

@@ -107,9 +107,16 @@ catalogs:
     install_policy: discovery-only
 ```
 
-Supported bundle source schemes are `https`, localhost `http`, `file`,
-`builtin`, and local filesystem paths. The other five catalog types accept
-remote HTTPS URLs, with HTTP allowed only for localhost.
+Supported bundle source schemes are `https`, `file`, `builtin`, and local
+filesystem paths. The other five catalog types accept remote HTTPS URLs.
+All six types reject plain HTTP catalog sources, including localhost and
+loopback addresses, before network access. This intentionally removes the
+former local-development HTTP exception so every source layer follows the
+same HTTPS policy. Local HTTPS development servers need a certificate trusted
+by Python's TLS client using its default OpenSSL trust configuration, which may
+differ from browser or operating-system trust. `SSL_CERT_FILE` can select a PEM
+CA bundle containing the development CA. Archive download policies are separate
+and unchanged.
 
 ## Configuration semantics
 
@@ -125,12 +132,17 @@ phase and accepted values otherwise differ:
   while `remove_source()` intentionally filters raw mappings by ID or URL
   without validating their other fields. This lets users remove a hand-edited
   invalid source instead of being trapped by the invalid entry.
+- Workflow and workflow-step catalog removal retains configuration-shape and
+  priority validation, but skips URL validation. Removal never fetches a source,
+  so legacy HTTP entries do not prevent removing either that source or an
+  unrelated HTTPS entry. Discovery and fetching still require HTTPS. Removal
+  retains priority order, with YAML declaration order breaking ties.
 - All readers reject boolean priorities. Replacement-layer loaders otherwise
   call `int(value)`, so a fractional numeric value such as `1.9` is truncated
   to `1`. Bundle `CatalogSource` materialization rejects booleans and floats
   rather than coercing them, but `_read()` and `remove_source()` can carry or
   remove those raw values without materializing them.
-- Replacement-layer loaders reject malformed remote URLs and non-localhost
+- Replacement-layer loaders reject malformed remote URLs and all plain
   HTTP while resolving the active stack. Bundle source validation depends on
   the path: catalog commands validate supported schemes and remote security
   before writing, while a hand-edited stack is revalidated by the fetcher when
@@ -171,12 +183,11 @@ Current behavior differs by primitive:
 - Extensions and presets use one-hour per-URL caches, retain legacy cache names
   for the official source, validate every redirect hop, validate cached payload
   shape, and treat cache read/write failures as best-effort.
-- Integrations use one-hour per-URL caches. The shared HTTP redirect handler
-  applies its strict scheme, host, and remote-to-local policy on every hop;
-  integrations do not add their catalog-specific URL validator per hop, but do
-  apply it to a changed terminal URL. They validate cached payload shape,
-  discard invalid cache entries when possible, and treat cache writes as
-  best-effort.
+- Integrations use one-hour per-URL caches and apply their catalog-specific URL
+  validator on every redirect hop and to a changed terminal URL. The shared HTTP
+  redirect handler also applies its strict scheme, host, and remote-to-local
+  policy on every hop. Integrations validate cached payload shape, discard
+  invalid cache entries when possible, and treat cache writes as best-effort.
 - Workflows use one-hour per-URL caches, validate every redirect hop, fall back
   to any readable stale cache after errors while opening, redirecting, bounded
   reading, UTF-8 decoding, or JSON parsing, and currently require only a mapping
@@ -227,7 +238,10 @@ usable. Validation errors are not uniformly recoverable:
   an `IntegrationCatalogError`, so the merge warns, skips that source, and can
   expose a lower-precedence copy.
 - Preset lookup preserves the first fetch error when no source is readable;
-  preset search currently converts ordinary fetch failures to an empty result.
+  preset search converts recoverable fetch failures, including rejected
+  redirects, to an empty result when no source is readable, regardless of source
+  order. Invalid source configuration (`PresetValidationError`) is fatal in
+  search and is validated before fetching, including rejected HTTP catalog URLs.
   `PresetCatalogValidationError` remains fatal: search can fail on invalid
   lower-precedence content even after reading a valid higher source, while a
   targeted lookup can return as soon as it finds the requested higher entry.

design/extension.md

@@ -346,8 +346,8 @@ packaging and output writes; they are not a sandbox.
 
 Network and package trust rules are:
 
-- catalog and archive URLs require HTTPS, except loopback HTTP for local
-  development;
+- catalog URLs require HTTPS, including for local development; archive URLs
+  retain the loopback HTTP exception;
 - catalog redirects are validated at every hop;
 - configured authentication providers may add credentials for matching hosts,
   and redirect handling does not forward them to untrusted hosts;

docs/bundles/catalogs.md

@@ -81,8 +81,9 @@ CLI add/remove operations write project scope only. Both readers accept a
 missing schema version for compatibility and reject an unsupported major
 version.
 
-Source URLs can be HTTPS, HTTP on localhost, `file://`, local paths, or known
-`builtin://` URLs. Added local paths are stored as absolute paths. This
+Source URLs can be HTTPS, `file://`, local paths, or known
+`builtin://` URLs. Plain HTTP is rejected before network access, including on
+localhost. Added local paths are stored as absolute paths. This
 flexibility applies to the catalog file, not its bundle download URLs.
 
 ### Precedence

docs/extensions/api.md

@@ -353,7 +353,7 @@ entry = CatalogEntry(
 
 | Field | Type | Description |
 |-------|------|-------------|
-| `url` | `str` | Catalog URL (must use HTTPS, or HTTP for localhost) |
+| `url` | `str` | Catalog URL (must use HTTPS, including for localhost) |
 | `name` | `str` | Human-readable catalog name |
 | `priority` | `int` | Sort order (lower = higher priority, wins on conflicts) |
 | `install_allowed` | `bool` | Whether extensions from this catalog can be installed |

docs/extensions/catalogs.md

@@ -179,9 +179,10 @@ Create a `catalog.json` file containing the releases your team has vetted:
 | GitHub Pages | `https://your-org.github.io/spec-kit-catalog/catalog.json` | Public or org-visible |
 | Internal web server | `https://internal.company.com/spec-kit/catalog.json` | Corporate network |
 | S3/cloud storage | `https://s3.amazonaws.com/your-bucket/catalog.json` | Cloud-hosted teams |
-| Local file server | `http://localhost:8000/catalog.json` | Development/testing |
+| Local HTTPS server | `https://localhost:8443/catalog.json` | Development/testing with a certificate trusted by Python |
 
-URLs must use HTTPS, except localhost for testing.
+Catalog URLs must use HTTPS, including localhost. Plain HTTP catalogs are
+rejected before network access.
 Add your catalog with `specify extension catalog add`, or use:
 
 ```yaml
@@ -236,14 +237,14 @@ inside the network. For example, an entry can use
 
 ### Development and testing
 
-```bash
-python -m http.server 8000 --directory ./my-catalog/
-```
-
-In another terminal:
+Serve `./my-catalog/` through an HTTPS server with a certificate trusted by
+Python's TLS client, not just your browser or operating system. For a development
+CA, point `SSL_CERT_FILE` at a PEM CA bundle containing that CA before running
+the CLI (see [TLS certificates](../reference/authentication.md#tls-certificates)):
 
 ```bash
-export SPECKIT_CATALOG_URL="http://localhost:8000/catalog.json"
+export SSL_CERT_FILE="/absolute/path/to/ca-bundle.pem"
+export SPECKIT_CATALOG_URL="https://localhost:8443/catalog.json"
 specify extension add my-new-extension
 ```
 

docs/integrations/cli.md

@@ -335,7 +335,9 @@ specify integration catalog add <url>
 | --------------- | ----------------------------- |
 | `--name <name>` | Optional name for the catalog |
 
-Adds a custom catalog URL to the project's `.specify/integration-catalogs.yml`. The URL must use HTTPS (except `http://localhost`, `http://127.0.0.1`, or `http://[::1]` for local testing).
+Adds a custom catalog URL to the project's `.specify/integration-catalogs.yml`.
+The URL must use HTTPS, including for local testing. Plain HTTP is rejected
+before network access, including on localhost and loopback addresses.
 
 Re-adding the same URL with the same name succeeds without changing the configuration; a different name is rejected.
 

docs/presets/catalogs.md

@@ -61,7 +61,7 @@ catalogs:
 
 | Entry field | Meaning |
 | --- | --- |
-| `url` | HTTPS catalog URL; loopback HTTP is allowed for local development |
+| `url` | HTTPS catalog URL, including for local development; plain HTTP is rejected |
 | `name` | Source name displayed by the CLI and used for provenance |
 | `priority` | Lower numbers are checked first |
 | `install_allowed` | Whether ID-based installation is permitted; default `false` |

docs/reference/authentication.md

@@ -343,6 +343,26 @@ You can configure multiple entries for different hosts or organizations:
    target leaves the entry's declared hosts — preventing credential
    leakage to CDNs or third-party services.
 
+## TLS certificates
+
+HTTPS catalog requests use Python's `urllib` and `ssl` certificate verification.
+The trusted certificate authorities depend on the Python distribution and its
+OpenSSL configuration. Trusting a development CA in a browser or macOS Keychain
+alone does not necessarily make it trusted by the Python interpreter running
+Specify.
+
+For a private or local development CA, set `SSL_CERT_FILE` to the absolute path
+of a PEM CA bundle containing that CA before running Specify:
+
+```bash
+export SSL_CERT_FILE="/absolute/path/to/ca-bundle.pem"
+```
+
+This selects a CA file rather than appending to Python's default CA file. Keep
+any other required trust roots in the bundle, including those needed for public
+catalogs and downloads. The server certificate must still be valid for the
+requested hostname or IP address. Keep certificate verification enabled.
+
 ## Template
 
 A reference `auth.json` with GitHub pre-configured:

docs/workflow-steps/catalogs.md

@@ -39,7 +39,8 @@ catalogs:
     description: Our vetted workflow steps
 ```
 
-`url` requires HTTPS (loopback HTTP is allowed for development). Lower numeric
+`url` requires HTTPS, including for local development; plain HTTP is rejected
+before network access. Lower numeric
 priorities win; the first source wins ties. `install_allowed` defaults to false
 in hand-authored configuration.
 
@@ -49,6 +50,28 @@ allowed and a priority after existing entries. It does not expose preset-style
 configuration when a discovery-only source is intended. Adding a project stack
 replaces implicit defaults unless you include them explicitly.
 
+### Migrating legacy HTTP sources
+
+Older configurations may contain loopback HTTP URLs that are no longer accepted.
+Remove a legacy project source with `specify workflow step catalog remove <index>`;
+removal does not fetch or validate source URLs. Indexes follow ascending
+priority, with YAML declaration order breaking ties; entries without a URL are
+skipped. `step catalog list` still rejects invalid URLs, so inspect
+`.specify/step-catalogs.yml` to determine the index when it cannot list them.
+Other entries can also be removed while a legacy HTTP source remains.
+
+Alternatively, edit `.specify/step-catalogs.yml` for project sources or
+`~/.specify/step-catalogs.yml` for user sources. Replace the source's `url` with
+a working HTTPS catalog URL, or remove its entire item from `catalogs`.
+Local HTTPS servers need a certificate trusted by Python's TLS client; browser
+or operating-system trust alone may not suffice. See
+[TLS certificates](../reference/authentication.md#tls-certificates) for
+`SSL_CERT_FILE` configuration. Changing the URL scheme alone does not enable HTTPS.
+
+An empty or deleted configuration falls through to the next level. Built-in
+defaults apply when no higher level supplies a stack. Also update or unset
+`SPECKIT_STEP_CATALOG_URL` if it still selects an HTTP source.
+
 ## Step entry schema
 
 Catalog JSON contains `schema_version` and a `steps` collection. Entries identify

docs/workflows/catalogs.md

@@ -37,10 +37,34 @@ catalogs:
     description: Our vetted workflows
 ```
 
-`url` requires HTTPS (loopback HTTP is allowed for development). Lower numeric priorities win; the first source wins ties. `install_allowed` defaults to false in hand-authored configuration.
+`url` requires HTTPS, including for local development; plain HTTP is rejected
+before network access. Lower numeric priorities win; the first source wins ties.
+`install_allowed` defaults to false in hand-authored configuration.
 
 `specify workflow catalog add` creates project entries with installation allowed and a priority after existing entries. It does not expose preset-style `--install-allowed` or `--priority` flags. Add only vetted sources; edit the configuration when a discovery-only source is intended. Adding a project stack replaces implicit defaults unless you include them explicitly.
 
+### Migrating legacy HTTP sources
+
+Older configurations may contain loopback HTTP URLs that are no longer accepted.
+Remove a legacy project source with `specify workflow catalog remove <index>`;
+removal does not fetch or validate source URLs. Indexes follow ascending
+priority, with YAML declaration order breaking ties; entries without a URL are
+skipped. `catalog list` still rejects invalid URLs, so inspect
+`.specify/workflow-catalogs.yml` to determine the index when it cannot list them.
+Other entries can also be removed while a legacy HTTP source remains.
+
+Alternatively, edit `.specify/workflow-catalogs.yml` for project sources or
+`~/.specify/workflow-catalogs.yml` for user sources. Replace the source's `url`
+with a working HTTPS catalog URL, or remove its entire item from `catalogs`.
+Local HTTPS servers need a certificate trusted by Python's TLS client; browser
+or operating-system trust alone may not suffice. See
+[TLS certificates](../reference/authentication.md#tls-certificates) for
+`SSL_CERT_FILE` configuration. Changing the URL scheme alone does not enable HTTPS.
+
+An empty or deleted configuration falls through to the next level. Built-in
+defaults apply when no higher level supplies a stack. Also update or unset
+`SPECKIT_WORKFLOW_CATALOG_URL` if it still selects an HTTP source.
+
 ## Workflow entry schema
 
 Catalog JSON contains `schema_version` and a `workflows` mapping keyed by workflow ID. Entries carry `id`, `name`, `description`, `author`, `version`, `url`, and optional `sha256`, `requires`, `repository`, `license`, `tags`, and timestamps. A download can be raw YAML or a complete workflow archive. First-party entries marked `bundled: true` use packaged definitions rather than a network download.

pyproject.toml

@@ -61,6 +61,7 @@ packages = ["src/specify_cli"]
 
 [project.optional-dependencies]
 test = [
+    "cryptography",
     "pytest>=7.0",
     "pytest-cov>=4.0",
     "pytest-xdist>=3.0",

src/specify_cli/bundles/adapters.py

@@ -92,7 +92,7 @@ def _file_url_to_path(parsed: ParseResult) -> Path:
 
 
 def _validate_remote_url(source_id: str, url: str) -> None:
-    """Restrict remote catalogs to HTTPS (HTTP only for localhost) with a host.
+    """Restrict remote catalogs to HTTPS with a host, including localhost.
 
     Mirrors ``specify_cli.catalogs`` URL validation to avoid MITM/downgrade
     issues before any network call.
@@ -110,11 +110,9 @@ def _validate_remote_url(source_id: str, url: str) -> None:
         raise BundlerError(
             f"Catalog '{source_id}' URL is malformed: {url}"
         ) from None
-    is_localhost = hostname in ("localhost", "127.0.0.1", "::1")
-    if parsed.scheme != "https" and not (parsed.scheme == "http" and is_localhost):
+    if parsed.scheme != "https":
         raise BundlerError(
-            f"Catalog '{source_id}' URL must use HTTPS (got {parsed.scheme}://). "
-            "HTTP is only allowed for localhost."
+            f"Catalog '{source_id}' URL must use HTTPS (got {parsed.scheme}://)."
         )
     # Check hostname, not netloc: netloc is truthy for host-less URLs like
     # "https://:8080" or "https://user@...", so requiring netloc would let

src/specify_cli/bundles/catalog_config.py

@@ -167,19 +167,17 @@ def add_source(
     if "://" in url and parsed.scheme.lower() not in _REMOTE_SCHEMES:
         raise BundlerError(
             f"Unsupported catalog url scheme '{parsed.scheme}://' in '{url}'. "
-            "Use http(s)://, file://, builtin://, or a local path."
+            "Use https://, file://, builtin://, or a local path."
         )
     if parsed.scheme.lower() in {"http", "https"}:
         # Mirror specify_cli.catalogs._validate_catalog_url (#3209/#3210):
-        # HTTPS only (HTTP just for localhost), and check hostname, not
+        # HTTPS only, including localhost, and check hostname, not
         # netloc — netloc is truthy for host-less URLs like "https://:8080"
         # or "https://user@". Validating here keeps junk out of
         # bundle-catalogs.yml instead of failing later at fetch time.
-        is_localhost = hostname in ("localhost", "127.0.0.1", "::1")
-        if parsed.scheme.lower() != "https" and not is_localhost:
+        if parsed.scheme.lower() != "https":
             raise BundlerError(
-                f"Catalog url must use HTTPS (got {parsed.scheme}://). "
-                "HTTP is only allowed for localhost."
+                f"Catalog url must use HTTPS (got {parsed.scheme}://)."
             )
         if not hostname:
             raise BundlerError(f"Catalog url must be a valid URL with a host: {url}")

src/specify_cli/catalogs.py

@@ -68,7 +68,7 @@ def _entry(
 
     @classmethod
     def _validate_catalog_url(cls, url: str) -> None:
-        """Validate that a catalog URL uses HTTPS, except localhost HTTP."""
+        """Validate that a catalog URL uses HTTPS, including local hosts."""
         from urllib.parse import urlparse
 
         try:
@@ -83,14 +83,12 @@ def _validate_catalog_url(cls, url: str) -> None:
             _ = parsed.port
         except ValueError:
             raise cls._error(f"Catalog URL is malformed: {url}") from None
-        is_localhost = hostname in ("localhost", "127.0.0.1", "::1")
-        if parsed.scheme != "https" and not (parsed.scheme == "http" and is_localhost):
+        if parsed.scheme != "https":
             # A scheme-less value (``example.com/catalog.json``) would otherwise read
             # "got ://". The value itself is not echoed: callers render errors as Rich markup.
             got = f"{parsed.scheme}://" if parsed.scheme else "a URL with no scheme"
             raise cls._error(
-                f"Catalog URL must use HTTPS (got {got}). "
-                "HTTP is only allowed for localhost."
+                f"Catalog URL must use HTTPS (got {got})."
             )
         # Check hostname, not netloc: netloc is truthy for host-less URLs like
         # "https://:8080" or "https://user@", so the host guarantee this error

src/specify_cli/integrations/__init__.py

@@ -357,7 +357,12 @@ def _fetch_single_catalog(
         try:
             from specify_cli.authentication.http import open_url
 
-            with open_url(entry.url, timeout=10) as resp:
+            def _validate_redirect(_old_url: str, new_url: str) -> None:
+                self._validate_catalog_url(new_url)
+
+            with open_url(
+                entry.url, timeout=10, redirect_validator=_validate_redirect
+            ) as resp:
                 # Validate final URL after redirects
                 final_url = resp.geturl()
                 if final_url != entry.url: