Skip to content
View kartsan03's full-sized avatar
  • 14:15 (UTC +01:00)

Block or report kartsan03

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
kartsan03/README.md

Tymur

I fix bugs in LLM libraries, inference servers and agent frameworks. Most of them turn out to be the same bug: a layer reports something other than what the layer under it did. I also build data pipelines, mostly for job-market data.

Building: wiretruth. Does your LLM SDK tell you the truth? It replays real provider responses through any-llm, LiteLLM, pydantic-ai, the Vercel AI SDK and TanStack AI, and checks what each one reports against what the provider sent. Results matrix.

Merged upstream work

94 merged PRs in 16 projects, counted in November 2026. The bigger ones:

Also embabel-agent (3), ollaya (3), any-llm (2), solana-py (2), and TanStack AI plus five more projects with one each. All of them.

Projects

  • ai-quality-gate: offline checks for recorded LLM output (schema, grounding, snapshots) that fail CI with an exit code and never call a model. On PyPI as aiqg.
  • actgate: an MCP proxy that holds an agent's tool calls until a person approves them and writes each decision to a hash-chained ledger. On PyPI as actgate.
  • djinni-market-etl: Djinni job postings in PostgreSQL, with one observation per posting per run, so time to close and reopens can be counted. Analysis recipes.
  • solana-sort-key-and-clock: which rule 16 Solana mainnet validators use to order transactions in a block, measured with my own probe transactions. The raw data and a one-command reproduction are in the repo.
  • StableInvoice: USDC escrow invoices on Solana. The client funds a vault up front, and the payment goes out in one sum once every milestone is accepted.

What I'm working out

How do you test code that sits between your program and a model provider, when the provider's answer changes from run to run? My current answer is to record real responses once and check everything against the recording. wiretruth does that for SDKs, ai-quality-gate for an app's own LLM output.

Pinned Loading

  1. wiretruth wiretruth Public

    Replays real LLM provider responses through LLM SDKs and checks whether each SDK reports what actually happened.

    Python 3

  2. ai-quality-gate ai-quality-gate Public

    Offline quality gate for recorded AI/LLM outputs: schema, grounding, snapshot and stability checks with CI exit codes. No API keys, no model calls.

    Python

  3. actgate actgate Public

    Local IntentLedger + MCP stdio proxy: propose, approve/deny, hash-chained ledger. Optional HMAC seals.

    Python