[rwzdk]

Problem

The v3 shared-preferences file is written with raw non-ASCII (ensure_ascii=False) but opened without an explicit encoding, so its charset silently depends on the machine locale:

astrbot/core/db/migration/shared_preferences_v3.py:20-29

with open(self.path) as f:
    return json.load(f)
...
with open(self.path, "w") as f:
    json.dump(self._data, f, indent=4, ensure_ascii=False)

Two consequences on Windows, where the default codec is the ANSI codepage (GBK/cp936 on Chinese Windows, cp1252 on Western):

1. Startup crash for a data directory migrated from another OS. A shared_preferences.json authored as UTF-8 (created on Linux/macOS, or edited by the user) that contains a character the local codepage cannot decode raises UnicodeDecodeError. _load_preferences only catches json.JSONDecodeError, and the module instantiates sp = SharedPreferences() at import time, which migra_3_to_4.py:14 imports — so the error escapes during the v3→v4 migration instead of being treated as a corrupt file.
2. A failed write destroys the file. open(self.path, "w") truncates before any encoding happens, so storing a value the codepage cannot encode raises mid-write and leaves the previously saved preferences gone or half-written.

Reproduced here on Chinese Windows (Python 3.12, locale.getpreferredencoding() = cp936):

E       UnicodeDecodeError: 'gbk' codec can't decode byte 0xab in position 22: illegal multibyte sequence
E       UnicodeEncodeError: 'gbk' codec can't encode character '\u2603' in position 1: illegal multibyte sequence
2 failed

On a machine whose locale already is UTF-8 the two tests pass even before the fix; they are correctness assertions rather than locale-sensitive probes, and the failure above is what Windows users hit.

Fix

Pin the charset for both directions and treat an undecodable file the same way a malformed one is already treated:

with open(self.path, encoding="utf-8") as f:
    ...
except (json.JSONDecodeError, UnicodeDecodeError):
    os.remove(self.path)
...
with open(self.path, "w", encoding="utf-8") as f:

This is the same convention the rest of the codebase uses for JSON text I/O (e.g. core/agent/runners/tool_loop_agent_runner.py:397, core/skills/neo_skill_sync.py:295, dashboard/services/skills_service.py:587 all pass encoding="utf-8"), and it matches AGENTS.md item 10 on cross-platform compatibility.

Tests

Two cases added to tests/unit/test_shared_preferences.py, next to the existing preference tests:

• reading a UTF-8 file that contains Japanese text and ☃ must return the exact value;
• put() of a value the OS codepage cannot encode must leave a valid file holding both the previous and the new key (the truncation case above).

After the change the whole file passes, including the pre-existing cases:

pytest tests/unit/test_shared_preferences.py -q
12 passed, 12 warnings in 5.50s

ruff format / ruff check are clean on both files. The fix touches only the v3 migration helper; the current DB-backed astrbot/core/utils/shared_preferences.py is unchanged.

Prepared with an AI coding agent.

Summary by Sourcery

Make legacy shared-preferences persistence reliably UTF-8 across platforms and protect existing data during non-ASCII writes.

Bug Fixes:

• Ensure v3 shared preferences are consistently read and written as UTF-8 across platforms.
• Treat undecodable preference files like malformed JSON by removing them instead of failing during migration.

Tests:

• Add coverage for reading non-ASCII UTF-8 preferences and preserving existing data when storing non-ASCII values.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/unit/test_shared_preferences.py" line_range="284" />
<code_context>
+
+    store = SharedPreferences(str(path))
+
+    assert store.get("greeting") == "こんにちは \u2603 snowman"
+
+
</code_context>
<issue_to_address>
**New tests pass paths as strings**

Both added tests convert `tmp_path` children to strings before passing them to `SharedPreferences`; its filesystem operations accept `Path` objects, so these new path-handling call sites do not follow the required `pathlib.Path` convention.

Pass `path` directly to `SharedPreferences` in both tests.

Also at `tests/unit/test_shared_preferences.py:296`.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. If an existing preferences file contains bytes that are not valid UTF-8, loading it now catches the decoding error and deletes the file, losing its saved preferences. Reverting stops future deletions but cannot restore files already removed.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook