Repositories list

• llm-d-inference-sim

  Public

  llm-d/llm-d-inference-sim’s past year of commit activity
  A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

  incubating

  incubating

  Go

  •

  Apache License 2.0

  •166166 forks•214214 stars•77 issues•77 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d-infra

  Public

  llm-d/llm-d-infra’s past year of commit activity
  repo for CI and infrastructure required to maintain llm-d org member repos

  Shell

  •

  Apache License 2.0

  •4242 forks•22 stars•2626 issues•1010 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d

  Public

  llm-d/llm-d’s past year of commit activity
  Achieve state of the art inference performance with modern accelerators on Kubernetes

  kubernetesaigpu+ 6

  kubernetesaigpucncfinferencemodel-serverllmdistributed-inferenceintelligent-routing

  Python

  •

  Apache License 2.0

  •850850 forks•4.8k4.8k stars•6565 issues•5656 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d-batch-gateway

  Public

  llm-d/llm-d-batch-gateway’s past year of commit activity
  llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine

  Go

  •

  Apache License 2.0

  •5858 forks•2727 stars•2626 issues•1616 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d-benchmark

  Public

  llm-d/llm-d-benchmark’s past year of commit activity
  llm-d benchmark scripts and tooling

  incubating

  incubating

  Python

  •

  Apache License 2.0

  •149149 forks•7373 stars•2424 issues•3030 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d-prism

  Public

  llm-d/llm-d-prism’s past year of commit activity
  Performance analysis for distributed inference systems

  JavaScript

  •

  Apache License 2.0

  •2525 forks•1414 stars•11 issue•00 pull requests•Updated Oct 15, 2026Oct 15, 2026

• llm-d-router

  Public

  llm-d/llm-d-router’s past year of commit activity
  llm-d Router: The intelligent entry point for inference requests

  kubernetesainetworking+ 2

  kubernetesainetworkinginferencegateway-api

  Go

  •

  Apache License 2.0

  •449449 forks•405405 stars•199199 issues•107107 pull requests•Updated Oct 14, 2026Oct 14, 2026

• llm-d-async

  Public

  llm-d/llm-d-async’s past year of commit activity
  Asynchronous Processor for Inference Gateway. Orchestrator of queues

  Go

  •

  Apache License 2.0

  •5656 forks•1515 stars•1616 issues•88 pull requests•Updated Oct 14, 2026Oct 14, 2026

• .project

  Public

  llm-d/.project’s past year of commit activity
  Project metadata for llm-d - CNCF .project automation

  00 forks•00 stars•11 issue•11 pull request•Updated Oct 14, 2026Oct 14, 2026

• llm-d-latency-predictor

  Public

  llm-d/llm-d-latency-predictor’s past year of commit activity
  Latency prediction service for ML-model based scoring with llm-d-inference-scheduler

  Python

  •

  Apache License 2.0

  •3535 forks•99 stars•1010 issues•3030 pull requests•Updated Oct 14, 2026Oct 14, 2026

• llm-d-kv-cache

  Public

  llm-d/llm-d-kv-cache’s past year of commit activity
  Distributed KV cache scheduling & offloading libraries

  aiincubating

  aiincubating

  Go

  •

  Apache License 2.0

  •164164 forks•186186 stars•1717 issues•2828 pull requests•Updated Oct 14, 2026Oct 14, 2026

• llm-d-autoscaling

  Public

  llm-d/llm-d-autoscaling’s past year of commit activity
  Variant optimization autoscaler for distributed inference workloads

  Go

  •

  Apache License 2.0

  •9797 forks•6060 stars•2020 issues•2020 pull requests•Updated Oct 13, 2026Oct 13, 2026

• llm-d-api-extensions

  Public

  llm-d/llm-d-api-extensions’s past year of commit activity
  Python

  •

  Apache License 2.0

  •22 forks•00 stars•00 issues•11 pull request•Updated Oct 12, 2026Oct 12, 2026

• llm-d.github.io

  Public

  llm-d/llm-d.github.io’s past year of commit activity
  Website for llm-d: This repository builds the website seen at llm-d.ai

  documentationopensourceai+ 1

  documentationopensourceaidocusaurus

  MDX

  •

  Apache License 2.0

  •7373 forks•1212 stars•1010 issues•1212 pull requests•Updated Oct 11, 2026Oct 11, 2026

• llm-d-inference-payload-processor

  Public

  llm-d/llm-d-inference-payload-processor’s past year of commit activity
  Inference payload processor for llm-d

  Go

  •

  Apache License 2.0

  •5454 forks•2323 stars•3535 issues•1717 pull requests•Updated Oct 9, 2026Oct 9, 2026

• llm-d-go-template

  Public template

  llm-d/llm-d-go-template’s past year of commit activity
  Go microservice template for llm-d repos. Use 'Use this template' to create a new Go project with standard CI, linting, Prow, and governance.

  Makefile

  •

  Apache License 2.0

  •88 forks•00 stars•11 issue•88 pull requests•Updated Oct 8, 2026Oct 8, 2026

• llm-d-pd-utils

  Public

  llm-d/llm-d-pd-utils’s past year of commit activity
  Python

  •1818 forks•66 stars•1111 issues•1313 pull requests•Updated Oct 8, 2026Oct 8, 2026

• llm-d-python-template

  Public template

  llm-d/llm-d-python-template’s past year of commit activity
  Python project template for llm-d repos. Use 'Use this template' to create a new Python project with standard CI, linting, Prow, and governance.

  Makefile

  •

  Apache License 2.0

  •77 forks•11 star•11 issue•44 pull requests•Updated Oct 8, 2026Oct 8, 2026

• .github

  Public

  llm-d/.github’s past year of commit activity
  88 forks•00 stars•11 issue•22 pull requests•Updated May 18, 2026May 18, 2026

• llm-d-routing-sidecar

  Public archive

  llm-d/llm-d-routing-sidecar’s past year of commit activity
  Incubating P/D sidecar for llm-d

  incubating

  incubating

  Go

  •

  Apache License 2.0

  •3434 forks•1717 stars•00 issues•00 pull requests•Updated Nov 13, 2025Nov 13, 2025

• llm-d-deployer

  Public archive

  llm-d/llm-d-deployer’s past year of commit activity
  Helm charts for llm-d

  helmhelm-chartllm-d

  helmhelm-chartllm-d

  Shell

  •

  Apache License 2.0

  •5757 forks•5252 stars•3434 issues•1717 pull requests•Updated Jul 22, 2025Jul 22, 2025

• llm-d-model-service

  Public archive

  llm-d/llm-d-model-service’s past year of commit activity
  Simplified model deployment on llm-d

  incubating

  incubating

  Go

  •

  Apache License 2.0

  •1919 forks•2828 stars•2929 issues•55 pull requests•Updated Jul 2, 2025Jul 2, 2025