Skip to content

add saveSlotState and restoreSlotState - #326

Merged
ngxson merged 1 commit into
masterfrom
xsn/slot_state
Nov 19, 2026
Merged

ngxson merged 1 commit into
masterfrom
xsn/slot_state

Conversation

@ngxson

@ngxson ngxson commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

close #298

Summary by CodeRabbit

  • New Features
    • Save and restore model slot state, including cached tokens, using binary snapshots.
    • Slot state operations default to slot 0 and report errors when saving or restoring fails.
  • Bug Fixes
    • Improved file tracking so only successfully opened model files are included.
  • Tests
    • Added coverage for saving and restoring slot state, including invalid snapshots.

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

No actionable comments were generated in the recent review.

Walkthrough

The change adds slot save and restore message types, native handlers that use a temporary file, and public TypeScript methods. The methods save slot state as bytes or restore bytes for a selected slot. Tests cover saved state, restored completions, and invalid restore data.

Changes: Slot State Operations

Layer / File(s) Summary
Slot operation message contracts
cpp/glue.hpp, src/glue/messages.ts
C++ and TypeScript define request and response messages for slot save and restore. The messages include slot IDs, status fields, token counts, and binary data where applicable.
Native slot save and restore
cpp/wllama.cpp, cpp/wllama-context.h, cpp/wllama-fs.h
The action dispatcher routes slot requests to native handlers. The handlers submit server tasks using /tmp/wllama_slot_state.bin, read or write state bytes, and return task results. The file wrapper records successful opens only for paths beginning with models/ or /models/.
Public slot state methods and validation
src/wllama.ts, src/wllama.test.ts
saveSlotState returns saved bytes, and restoreSlotState returns the restored token count. Both default to slot 0 and report failures as inference errors. Tests check state persistence, matching completions, and invalid restore handling.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant Wllama
  participant wllama_action
  participant wllama_context
  participant server_task
  participant temp_file
  Wllama->>wllama_action: Send slot save or restore request
  wllama_action->>wllama_context: Call the matching slot action
  wllama_context->>server_task: Submit slot task and run queue
  server_task->>temp_file: Save or load slot state
  wllama_context->>temp_file: Read saved bytes or write restore bytes
  wllama_context-->>Wllama: Return task result and response data
Loading

Suggested reviewers: thomas-0816

Merge Risk: Low · up to 8859c

This change adds methods to save and restore a model slot's state as bytes. Restoring a large saved state makes one extra full-size copy while the request is sent to the worker, which costs memory and time but does not break behavior. It is safe to merge, and transferring the encoded buffer can follow.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the two new public methods, saveSlotState and restoreSlotState, which are the main change in the changeset. The PR adds these methods to the Wllama class in src/wllama.ts, along with t…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title names the two new public methods, saveSlotState and restoreSlotState, which are the main change in the changeset. The PR adds these methods to the Wllama class in src/wllama.ts, along with the worker messages, the native slot save/restore actions, and the generated bindings that support them.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dc28a2d3-6ab3-4392-b90c-265ef0b02925
Commits

Reviewing files that changed from the base of the PR and between 7ff82ca and 392cc09.

Files selected for processing: 10

  • compat/wasm/wllama.js
  • cpp/glue.hpp
  • cpp/wllama-context.h
  • cpp/wllama-fs.h
  • cpp/wllama.cpp
  • src/glue/messages.ts
  • src/wasm/wllama.js
  • src/wllama.test.ts
  • src/wllama.ts
  • src/workers-code/generated.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@ngxson
ngxson merged commit fb07aa9 into master Nov 19, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Save/restore model state (KV + recurrent) to persist prompt prefixes across reloads

1 participant