All pull requests

Pull requests

• Use os.replace when moving the downloaded file into the cache (Windows FileExistsError race)

  #620·hazelxue opened Oct 16, 2026·

  ·

• fix(load): support basename cache lookup and TIKTOKEN_OFFLINE mode

  #619·reginaldalfret opened Oct 14, 2026·

  ·
  1

• fix(build): Link macOS wheels with -headerpad_max_install_names

  #617·johnp opened Oct 12, 2026·

  ·

• fix(model): recognize GPT-6 family model names

  #616·sylvesterkaczmarek opened Oct 9, 2026·

  ·
  2

• Preserve full regex matches in Python BPE helpers

  #614·sylvesterkaczmarek opened Oct 5, 2026·

  ·

• Reset encoding discovery after interrupted plugin imports

  #613·sylvesterkaczmarek opened Oct 5, 2026·

  ·

• Treat unreadable default cache entries as cache misses

  #612·qorud02 opened Oct 5, 2026·

  ·

• Fix concurrent cache initialization on Windows

  #611·codewithfourtix opened Oct 2, 2026·

  ·
  1

• Map fine-tuned gpt-4.1 models to o200k_base

  #610·jayzuccarelli opened Oct 2, 2026·

  ·

• Check the content hash when caching is disabled

  #609·Linxiushen opened Sep 28, 2026·

  ·

• Clean up the temporary file if writing to the cache fails

  #607·Linxiushen opened Sep 21, 2026·

  ·

• feat: add explicit encoding cache release

  #605·mehrdadxzaker opened Sep 12, 2026·

  ·
  3

• Replace transmute-based hash_current_thread with a TLS-backed atomic counter

  #603·Berat121 opened Sep 10, 2026·

  ·

• Speed up educational BPE training

  #600·Marchematics opened Aug 31, 2026·

  ·
  1

  5

• Handle Data Gym vocab files without trailing newline

  #598·tandede opened Aug 18, 2026·

  ·
  1

• Validate base64 tokens when loading BPE files

  #592·weivwang opened Aug 12, 2026·

  ·

• Add get_token_length methods to more cheaply calculate token lengths

  #591·thomasaarholt opened Aug 11, 2026·

  ·

• Add DisallowedSpecialTokenError (ValueError subclass) for disallowed special tokens (#290)

  #590·JayYarlagadda opened Aug 7, 2026·

  ·
  1

• Fix remaining panics from #400 in encode_ordinary/_encode_bytes/encode_with_unstable

  #587·yanmo42 opened Jul 30, 2026·

  ·

• fix: Unable to route GET requests through proxy

  #584·thiagobarbosa opened Jul 23, 2026·2 tasks done·

  ·
  1

• Avoid panic when byte_pair_encode is given an empty piece

  #583·devYRPauli opened Jul 20, 2026·

  ·
  1

• Handle surrogate pairs in encode_with_unstable

  #581·devYRPauli opened Jul 16, 2026·

  ·
  1

  7

• Add CI support for Windows ARM64

  #580·chinazhangchao opened Jul 14, 2026·

  ·
  6

• Return ValueError instead of panicking on duplicate mergeable_ranks

  #579·devYRPauli opened Jul 13, 2026·

  ·
  1

• fix: apply surrogate-pair fixup in encode_with_unstable

  #575·steps-re opened Jul 8, 2026·

  ·
  1