Skip to content

feat(conversations): optionally clean up UMO metadata when deleting conversations - #10816

Open
SX0YYYY wants to merge 1 commit into
AstrBotDevs:masterfrom
SX0YYYY:feat/conversation-delete-metadata-cleanup
Open

SX0YYYY wants to merge 1 commit into
AstrBotDevs:masterfrom
SX0YYYY:feat/conversation-delete-metadata-cleanup

Conversation

@SX0YYYY

@SX0YYYY SX0YYYY commented Oct 27, 2026 •

Copy link
Copy Markdown

Motivation

Closes #10704.

Deleting a conversation (single or batch) from Data & Logs → Conversations only removes the row from conversations, leaving orphaned umo_aliases / umo_senders rows that the UI offers no way to clean up. #10761 only covers deleting a whole bot, not an individual conversation.

Following the two-layer plan in #10704 (comment), this PR keeps the precise primitive and adds an explicit, default-off cleanup option at the service layer. Backward compatible: cleanup_metadata defaults to false on both single and batch delete paths.

Modifications

  • DB layer (astrbot/core/db/__init__.py, sqlite.py)
    • Add delete_umo_alias(umo) -> bool primitive.
    • Add delete_umo_metadata_if_unreferenced(umo) -> dict: deletes the UMO's alias and sender metadata only when no conversation references it. Both DELETEs carry the NOT EXISTS guard in the statement itself, inside one transaction, so metadata is not deleted while a committed conversation still references the UMO. Returns {alias_deleted, senders_deleted, retained}.
  • Service layer (conversation_service.py)
    • delete_conversation accepts cleanup_metadata (default False) for both single and batch paths.
    • Batch path collects the deleted UMOs and cleans them up after all CIDs are gone.
    • Returns structured metadata_cleanup: {deleted_aliases, deleted_senders, retained_umos}; UMOs kept because other conversations still reference them are listed instead of reporting a plain success.
    • Cleanup errors never fail the completed deletion; they are reported as a structured error field.
  • API (dashboard/api/conversations.py, schemas.py)
    • cleanup_metadata query parameter on DELETE /conversations/{id} and body field on batch delete; both default to false.
    • openspec/openapi-v1.yaml, docs/public/openapi.json and the generated dashboard SDK are updated.
  • Dashboard
    • New default-off checkbox "同时清理无对话引用的会话元数据(别名等)" in the delete dialogs of both pages: the workspace page (batch delete) and the legacy page (single and batch delete).
    • i18n added for zh-CN / en-US / ja-JP / ru-RU.
  • Changelog (changelogs/unreleased.md).

Screenshots or Test Results

Verification steps (local AstrBot 4.29.3):

  1. Data & Logs → Conversations: delete a conversation whose UMO has no other conversations, with the cleanup checkbox on; verify the UMO's rows in umo_aliases and umo_senders are gone (row counts before/after).
  2. Same UMO with two conversations: delete one with the checkbox on; verify the alias row is retained and reported under retained_umos.
  3. Batch-delete the remaining conversations of that UMO with the checkbox on; verify alias and sender rows are removed after the last CID.
  4. Repeat a single delete with the checkbox off; verify the previous behavior is unchanged (rows remain).

All four steps passed on local AstrBot 4.29.3 with sanitized test data. Screenshots are attached via GitHub's uploader (sanitized): the cleanup checkbox in the delete dialog, the list after cleanup, and same-UMO retention after deleting one of two conversations.

截图1_删除确认框_清理勾选项 截图2_清理后列表 截图3_同UMO删一个_别名保留

Tests:

  • tests/unit/test_conversation_metadata_cleanup.py: 19 tests (db / service / API)
  • full suite on this branch: 4488 passed / 41 failed / 79 skipped / 26 collection errors; on unmodified master dad255c with the same command: 4469 passed / 41 failed / 79 skipped / 26 collection errors — the 41 failures and 26 collection errors are pre-existing environment issues (per-file counts and commands in the first comment)
  • ruff check ., ruff format --check, vue-tsc, npm run build, i18n key check all pass

Checklist

  • If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
  • My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
  • WebUI docs are unaffected (optional checkbox inside existing delete dialogs; no navigation or label changes).
  • No new dependencies.
  • No malicious code.

Notes

  • conversations has no index on user_id; the NOT EXISTS subquery therefore scans the table. On a local SQLite database with 100,000 conversation rows, one guarded cleanup took ~4.8 ms median (p95 ~5.4 ms, 200 runs; measured with a local benchmark script included in the first comment). Conversation deletion is low-frequency; this PR does not change the schema.
  • Docs: checked docs/zh/use/webui.md and docs/en/use/webui.md; neither mentions alias cleanup on conversation deletion. Changelog entry included in changelogs/unreleased.md.
  • Relation to feat(db): add UMO alias deletion method #10746: this PR currently carries an equivalent delete_umo_alias primitive. If feat(db): add UMO alias deletion method #10746 lands first, this PR will be rebased and the duplicate primitive dropped; only the service/API/UI delta remains.

Summary by Sourcery

Enable users to optionally remove orphaned conversation metadata when deleting individual or multiple conversations while preserving the existing default behavior.

New Features:

  • Add an opt-in cleanup option to single and batch conversation deletion that removes unreferenced UMO alias and sender metadata.
  • Expose metadata cleanup controls in the conversation deletion APIs and dashboard dialogs.

Enhancements:

  • Preserve metadata for UMOs still referenced by other conversations and return structured cleanup results, including retained sessions and non-fatal errors.

Documentation:

  • Update the conversation deletion OpenAPI specifications, generated client types, and unreleased changelog.

Tests:

  • Add coverage for database, service, API forwarding, retention, batch behavior, concurrency, and cleanup error handling.

Chores:

  • Add localized cleanup controls to the conversation workspace and legacy dashboard views.

@SX0YYYY

SX0YYYY commented Oct 27, 2026

Copy link
Copy Markdown
Author

Pre-existing failure evidence — referenced in the PR description as "the first comment".

Evidence: the 41 full-suite failures are pre-existing on unmodified master

Two full-suite runs were executed with the exact same command, on the same machine and environment:

  • this branch (d51fa61): 4488 passed / 41 failed / 79 skipped / 26 collection errors (257.54s)
  • unmodified master (dad255c, worktree): 4469 passed / 41 failed / 79 skipped / 26 collection errors (254.88s)

The 41 failed test IDs are identical in both runs (diff of the two FAILED lists is empty); the only difference in passed count is +19, exactly the 19 tests added by this PR. All 26 collection errors are ModuleNotFoundError for optional dependencies, also identical in both runs.

  • Command (both runs): pytest tests/ -q --continue-on-collection-errors
  • Environment: Windows 10 x64, CPython 3.12 (project virtualenv)

Per-file failure counts (identical in both runs)

Test file Failed
FAILED tests/test_builtin_pdf_skill.py 2
FAILED tests/test_computer_fs_tools.py 1
FAILED tests/test_media_utils.py 5
FAILED tests/test_profile_aware_tools.py 5
FAILED tests/test_shipyard_neo_booter.py 1
FAILED tests/test_tool_loop_agent_runner.py 1
FAILED tests/unit/test_computer.py 20
FAILED tests/unit/test_faiss_vec_db.py 3
FAILED tests/unit/test_kb_upload_atomicity.py 3
Total 41

The 41 failing test IDs (same list in both runs)

FAILED tests/test_builtin_pdf_skill.py::test_builtin_pdf_skill_converts_limited_markdown
FAILED tests/test_builtin_pdf_skill.py::test_builtin_pdf_skill_rejects_unsupported_default_font_character
FAILED tests/test_computer_fs_tools.py::test_file_read_tool_returns_image_call_tool_result_for_images
FAILED tests/test_media_utils.py::test_tencent_silk_encoding_uses_pysilk_tencent_format[24k-mono]
FAILED tests/test_media_utils.py::test_tencent_silk_encoding_uses_pysilk_tencent_format[44.1k-mono]
FAILED tests/test_media_utils.py::test_tencent_silk_encoding_uses_pysilk_tencent_format[22.05k-mono]
FAILED tests/test_media_utils.py::test_tencent_silk_encoding_uses_pysilk_tencent_format[48k-stereo]
FAILED tests/test_media_utils.py::test_tencent_silk_encoding_uses_pysilk_tencent_format[44.1k-stereo]
FAILED tests/test_profile_aware_tools.py::TestResolveProfile::test_selects_browser_profile
FAILED tests/test_profile_aware_tools.py::TestResolveProfile::test_falls_back_to_default_on_api_error
FAILED tests/test_profile_aware_tools.py::TestResolveProfile::test_falls_back_on_empty_profiles
FAILED tests/test_profile_aware_tools.py::TestResolveProfile::test_single_profile_selected
FAILED tests/test_profile_aware_tools.py::TestResolveProfile::test_auth_error_not_silenced
FAILED tests/test_shipyard_neo_booter.py::TestGetBooterRebuild::test_stale_non_neo_booter_calls_plain_shutdown
FAILED tests/test_tool_loop_agent_runner.py::test_tool_result_includes_all_calltoolresult_content
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_unwraps_bay_exec_response
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_keeps_flat_exec_response_compatible
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_maps_nonzero_return_code
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_failure_without_success_field
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_keeps_envelope_error
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_file_tools_unwrap_bay_exec_response
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_remaining_file_tools_unwrap_bay_exec_response[create_file-kwargs0-inner0]
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_remaining_file_tools_unwrap_bay_exec_response[write_file-kwargs1-inner1]
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_remaining_file_tools_unwrap_bay_exec_response[delete_file-kwargs2-inner2]
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_remaining_file_tools_unwrap_bay_exec_response[edit_file-kwargs3-inner3]
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_envelope_failure_wins_over_data_success
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_shell_envelope_failure_without_data
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_booter_init
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_booter_boot
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_available_healthy
FAILED tests/unit/test_computer.py::TestShipyardBooter::test_shipyard_available_unhealthy
FAILED tests/unit/test_computer.py::TestBoxliteBooter::test_boxlite_booter_init
FAILED tests/unit/test_computer.py::TestComputerClient::test_get_booter_shipyard
FAILED tests/unit/test_computer.py::TestComputerClient::test_get_booter_reuses_existing
FAILED tests/unit/test_computer.py::TestComputerClient::test_get_booter_rebuild_unavailable
FAILED tests/unit/test_faiss_vec_db.py::test_embedding_storage_rejects_zero_dimension_for_a_fresh_index
FAILED tests/unit/test_faiss_vec_db.py::test_embedding_storage_rejects_negative_dimension_for_a_fresh_index
FAILED tests/unit/test_faiss_vec_db.py::test_embedding_storage_accepts_a_valid_dimension_for_a_fresh_index
FAILED tests/unit/test_kb_upload_atomicity.py::test_insert_batch_real_storage_rolls_back_on_faiss_failure
FAILED tests/unit/test_kb_upload_atomicity.py::test_insert_batch_real_storage_dimension_mismatch_leaves_no_docs
FAILED tests/unit/test_kb_upload_atomicity.py::test_cleanup_failed_upload_real_vec_db_by_kb_doc_id

Collection errors (26, identical in both runs)

All are ModuleNotFoundError while importing test modules — optional platform/dependency SDKs not installed in this local environment, unrelated to this PR:

  • aiocqhttp x4
  • botpy x4
  • dingtalk_stream x1
  • discord x1
  • docx x1
  • lark_oapi x5
  • markitdown_no_magika x1
  • qrcode x2
  • slack_sdk x5
  • telegram x1
  • xinference_client x1

Full raw logs are retained locally (pytest_d51fa61_full_20261027.log, pytest_dad255c_baseline_20261027.log).


Benchmark script — referenced in the Notes section of the PR description:

"""Benchmark: guarded cleanup DELETE with NOT EXISTS, conversations without a user_id index.

Reproduces the number quoted in the PR Notes:
- 100,000 conversation rows, no user_id index
- per-cleanup median / p95 over 200 runs (SQLite, WAL)

Run: python 基准_索引notexists_benchmark.py
"""
import os
import sqlite3
import statistics
import tempfile
import time


def main() -> None:
    n_rows = 100_000
    n_umos = 1000
    runs = 200

    db = os.path.join(tempfile.gettempdir(), "bench_cleanup.db")
    for suffix in ("", "-wal", "-shm"):
        try:
            os.remove(db + suffix)
        except FileNotFoundError:
            pass

    con = sqlite3.connect(db)
    cur = con.cursor()
    cur.execute("PRAGMA journal_mode=WAL")
    cur.execute(
        "CREATE TABLE conversations ("
        "inner_conversation_id INTEGER PRIMARY KEY, user_id TEXT, "
        "created_at DATETIME, updated_at DATETIME)"
    )
    cur.execute("CREATE TABLE umo_aliases (id INTEGER PRIMARY KEY, umo TEXT UNIQUE)")
    cur.execute("CREATE TABLE umo_senders (id INTEGER PRIMARY KEY, umo TEXT)")
    cur.executemany(
        "INSERT INTO conversations (inner_conversation_id,user_id,created_at,updated_at) "
        "VALUES (?,?,?,?)",
        (
            (i, f"qq:GroupMessage:{i % n_umos}", "2026-10-26", "2026-10-26")
            for i in range(n_rows)
        ),
    )
    cur.executemany(
        "INSERT INTO umo_aliases (umo) VALUES (?)",
        ((f"orphan:{i}",) for i in range(runs)),
    )
    con.commit()

    plan = cur.execute(
        "EXPLAIN QUERY PLAN DELETE FROM umo_aliases WHERE umo=? "
        "AND NOT EXISTS (SELECT 1 FROM conversations WHERE user_id=?)",
        ("x", "x"),
    ).fetchall()
    print("plan:", plan)

    times = []
    for i in range(runs):
        umo = f"orphan:{i}"
        t0 = time.perf_counter()
        cur.execute(
            "DELETE FROM umo_aliases WHERE umo=? "
            "AND NOT EXISTS (SELECT 1 FROM conversations WHERE user_id=?)",
            (umo, umo),
        )
        times.append((time.perf_counter() - t0) * 1000)
        con.commit()

    print(
        f"rows={n_rows} runs={runs} "
        f"median_ms={statistics.median(times):.3f} "
        f"p95_ms={sorted(times)[int(runs * 0.95)]:.3f} "
        f"min_ms={min(times):.3f} max_ms={max(times):.3f}"
    )
    con.close()


if __name__ == "__main__":
    main()

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/dashboard/services/conversation_service.py" line_range="353-356" />
<code_context>
+        Conversations are already deleted when this runs, so a cleanup error
+        is reported as a structured field instead of a failed request.
+        """
+        try:
+            return await self._cleanup_orphaned_metadata(umos)
+        except Exception as exc:
+            logger.warning(f"清理会话元数据失败: {exc!s}")
+            return {
+                "deleted_aliases": 0,
</code_context>
<issue_to_address>
**Cleanup error handling is extracted**

When metadata cleanup raises an exception, `_safe_cleanup_metadata` handles the cleanup exception and logs it outside the single- and batch-delete main function blocks, contrary to the required placement of error handling and logging.

Handle the cleanup exception and logging directly in the single- and batch-delete main function blocks.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. With cleanup enabled, alias and sender metadata rows for sessions without remaining conversations are permanently deleted; reverting the code will not restore customized values. The deletion is opt-in and limited to those sessions, so the unrecoverable loss is bounded.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment on lines +353 to +356
try:
return await self._cleanup_orphaned_metadata(umos)
except Exception as exc:
logger.warning(f"清理会话元数据失败: {exc!s}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Low · Cleanup error handling is extracted

When metadata cleanup raises an exception, _safe_cleanup_metadata handles the cleanup exception and logs it outside the single- and batch-delete main function blocks, contrary to the required placement of error handling and logging.

Handle the cleanup exception and logging directly in the single- and batch-delete main function blocks.

Prompt for AI agents
In `astrbot/dashboard/services/conversation_service.py` at lines 353-356:

**Cleanup error handling is extracted**

When metadata cleanup raises an exception, `_safe_cleanup_metadata` handles the cleanup exception and logs it outside the single- and batch-delete main function blocks, contrary to the required placement of error handling and logging.

Handle the cleanup exception and logging directly in the single- and batch-delete main function blocks.

From AGENTS.md L73:

Handle edge cases, error catching, and logging directly inside the main function block instead of offloading them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 提供 delete_umo_alias 官方方法,避免会话删除后别名数据成为孤儿

1 participant