Shared L2 server e2e#
LMCache and pytest run as ordinary Python processes in the current environment.
The shared case starts a real lmcache server and uses CPU KV buffers through
LMCache dev’s ServerBenchClient, without an inference engine or GPU. Shared
cases and the test runner do not invoke Docker or Compose.
Layers#
CI integration tests live under .github/e2e/; ordinary unit tests remain
under tests/. Each backend owns one test directory and one guide in this
documentation section.
tests/ Ordinary UT and adapter tests
.github/e2e/conftest.py Shared server fixture, readiness and cleanup
.github/e2e/common_scripts/
cases.py L2Backend contract and correctness assertions
run.sh Native pytest command and artifact locations
.github/e2e/<plugin>/
test_server_roundtrip.py Backend fixture and shared case invocation
service.sh Backend-specific start/logs/stop commands
Default pytest collects only tests/; e2e uses .github/e2e/pytest.ini.
L2Backend supplies adapter_config (the server’s --l2-adapter JSON),
status_type (its reported adapter type), and exists(keys) (an independent
backend probe returning one boolean per LMCache ObjectKey, in input order).
Vendor fixtures own SDK clients, credentials and storage-key encoding.
The shared case stores four chunks of deterministic two-layer KV, independently confirms they reached L2, waits for asynchronous writes, clears L1 through HTTP and checks its object count is zero, then retrieves into zeroed CPU buffers and compares every chunk checksum.
Prepare a native Python environment#
Use Linux, Python 3.12, git and a C++ compiler, with enough memory to build LMCache’s CPU extensions. From the repository root (Autumn is the example plugin; substitute the chosen plugin’s extra):
python3.12 -m venv .venv-e2e
source .venv-e2e/bin/activate
python -m pip install uv
uv pip install torch --index-url https://download.pytorch.org/whl/cpu
LMCACHE_SRC=$(mktemp -d)
git clone https://github.com/LMCache/LMCache.git "$LMCACHE_SRC"
LMCACHE_REF=${LMCACHE_REF:-$(git ls-remote https://github.com/LMCache/LMCache.git refs/heads/dev | cut -f1)}
git -C "$LMCACHE_SRC" checkout "$LMCACHE_REF"
uv pip install -r "$LMCACHE_SRC/requirements/build.txt"
NO_GPU_EXT=1 MAX_JOBS=1 uv pip install --no-build-isolation "$LMCACHE_SRC"
uv pip install ".[autumn,test]" pytest-timeout
Start or configure the chosen backend as its README describes, then run:
PLUGIN=autumn
bash .github/e2e/common_scripts/run.sh ".github/e2e/$PLUGIN"
With an existing backend, set the fixture’s connection variables and run that command directly; backend provisioning scripts are not part of the case.
Add an L2 plugin#
Provide an
L2Backendfixture and request thelmcache_serverfixture and invokeassert_store_clear_retrieve(l2_backend, lmcache_server)in.github/e2e/<plugin>/test_server_roundtrip.py. Add.github/e2e/<plugin>/__init__.pyso tests use thee2e.<plugin>namespace instead of shadowing the vendor SDK.For CI, implement
.github/e2e/<plugin>/service.shwithstart,logs, andstop.startwaits for readiness and may write Bashexportstatements to$E2E_ARTIFACTS/backend.env; CI sources this file before running pytest.logswrites backend diagnostics to$E2E_ARTIFACTS;stopreleases only test-owned resources and tolerates a partially failed startup.Declare runtime and test dependencies in
lmcache_l2_plugins/<plugin>/plugin.toml, following the plugin guide. The existing UT gate discovers every declaration and verifies it in an isolated environment. E2e CI reuses that inventory and opts in plugins with complete.github/e2e/<plugin>/folders; no workflow matrix or second plugin list is edited.
A registered plugin without an e2e folder runs UT only. Partial e2e folders or folders for undeclared plugins fail discovery rather than silently skipping. The e2e pytest configuration uses importlib mode and package namespaces so different vendors can share test filenames when their directories are collected together.
Backend startup is entirely vendor-owned: native processes, Compose, or an existing remote service can all be used. No Compose file or container image is required by the shared interface. Only Autumn’s service script currently uses Compose, exclusively for its storage cluster.
CI and evidence#
The Package workflow builds and verifies the wheel and source distribution once.
Its e2e jobs depend on that build and download the same run’s distributions
artifact. They install the wheel with the selected plugin extra, then copy only
.github/e2e/ outside the checkout before running tests. This prevents source imports
from masking an incorrect or incomplete wheel. The wheel checksum is recorded
with the evidence. Local source installation above is a development shortcut;
to reproduce CI, install the downloaded wheel with [autumn,test] instead.
GitHub-hosted Ubuntu runners execute LMCache and pytest directly for all PRs (including drafts), nightly at 02:17 Asia/Shanghai, published releases and prereleases, and manual dispatch. Release runs test their tag. Each run records exact plugin and LMCache dev revisions and installed package versions.
Artifacts include JUnit XML, LMCache logs and backend diagnostics as
l2-server-e2e-<plugin>. Local output defaults to e2e-artifacts;
E2E_ARTIFACTS overrides it. Cleanup and log collection run after test failure.
lmcache bench l2 directly exercises an adapter; this case covers the real
server protocol, asynchronous store, L1 clear, L2 prefetch and retrieval.