Training and fine-tuning
SharpMind has three training jobs. They share the data pipeline, checkpoints and export, but they train different things and use different devices.
Three modes
| Mode | Starts from | Trains | GPU |
|---|---|---|---|
| New model (pretrain) | Random weights | Every weight | No — CPU only |
| Improve (LoRA) | Packaged .smm | Small adapters | Yes, when the base fits |
| SFT | Packaged .smm + chat JSONL | LoRA adapters, assistant tokens only | Yes, same shape rules |
GPU support
The GPU training engine accepts a base model with all of these:
- RMSNorm (not LayerNorm) — both in the blocks and the final norm.
- RoPE or NoPE positional encoding.
- Gated FFN (SwiGLU/GeGLU-style), not MoE.
- An attention layout with q_proj, k_proj, v_proj, o_proj LoRA targets.
- A packaged .smm base. Convert a GGUF first.
LLaMA/Qwen-style bases satisfy this. GPT-2/BERT-style bases (LayerNorm, dense GELU) do not for training, although GPU inference now covers them.
On a supported base the log shows:
Fine-tune: 8 adapters (rank 8, alpha 16).
Engine: [Cuda] NVIDIA A16, 14962 MB, cuBLAS 12.6
Training 100 steps (float32) — full backprop (loss → gradients → AdamW)…
On an unsupported base it shows the refusal and falls back:
Accelerator 'ilgpu' declined: GPU engine (M1) does not support a
LayerNormLayer final norm — LayerNorm, only RMSNorm
Retrying on the CPU. Set SHARPMIND_ACCELERATOR_FALLBACK=false to fail instead.
Studio's source build copies the optional SharpMind.GPU plugin into plugins/ilgpu with its ILGPU dependencies. The Accelerator list then offers ilgpu beside the built-in CPU. The plugin needs a CUDA or OpenCL device; without one it declines and the CPU path runs. The plugin is not a NuGet package, so a packaged install only gets it when that folder is shipped.
The fallback is a host policy. Set SHARPMIND_ACCELERATOR_FALLBACK=false to fail the job instead of retrying.
Fine-tune a base
- Training → Improve.
- Base model: pick a packaged .smm. The model list is under Models; a GGUF must be converted first.
- Corpus: use Browse. With SFT off it filters *.txt; with SFT on, *.jsonl. A folder or glob also works.
- LoRA: rank 8–16, alpha 16–32, targets q_proj k_proj v_proj o_proj.
- Steps: 50–200 for a tiny test, 200–500+ for a real SFT run.
- Output: merged .smm, adapter .smlora, or both.
- Run on: This PC or a Compute host. The base and corpus upload with the job; the export downloads back.
- Start fine-tune and watch the engine line and loss curve.
SFT
Tick Instruction tuning (SFT) to train on chat JSONL with assistant-only loss. Each line is one record. Supported shapes:
{"messages":[{"role":"user","content":"…"},{"role":"assistant","content":"…"}]}
{"conversations":[{"from":"human","value":"…"},{"from":"gpt","value":"…"}]}
{"prompt":"…","response":"…"}
{"instruction":"…","input":"…","output":"…"}
The base supplies the tokenizer and chat template. Use Preview data before the run: it reports parsed, skipped, truncated and train/val/exam counts, plus sample rendered conversations.
- Blank lines, malformed JSON and records without an assistant turn are skipped.
- Long conversations are truncated to the SFT sequence length.
- FuseChat's original file is a root JSON array, not JSONL; convert one record per line first.
Choosing data
| Data | Use |
|---|---|
| Plain .txt | Plain LoRA fine-tune (SFT off). Shakespeare, stories, domain text. |
| Text JSONL ({"text":"…"}) | Not for SFT — no assistant turn. Add it with the JSONL source so each record's text is one document. |
| Chat JSONL | SFT. Use messages, conversations, prompt/response or instruction/output. |
| Parquet | Not read by Studio. Convert to JSONL first. |
A JSONL file must go through the JSONL source. A text source reads the raw {"text":"…"} line including its JSON escapes, so the model learns to write \u0022 instead of quotes. The quick Add a text or JSONL corpus box picks the right source for a .jsonl path.
The job editor's Randomize button next to Seed changes the split, batching order and weight initialisation in one click.
For a quick test, a 1 MB text file is enough. For a useful instruction tune, a few thousand chat records are enough to see behaviour; 20k+ is a small real run. Prefer data that matches how you will use the model.
Remote runs
Select a Compute host in Run on. Studio uploads the source files and base, starts the job, polls its state and logs, then downloads the export back to the path shown on the form. The host needs no chat model for a fine-tune.
- A GPU run needs a vcg- plan and the host's GPU host checkbox ticked; a vc2- plan runs on the CPU. See Vultr + SSH.
- If the base is large, upload time dominates the first poll; the card logs progress in 25% steps.
- The export is written under the host's /data/SharpMind/Training/jobs/…, then downloaded.
- Rehearse 30 steps runs locally and estimates step time; it is not the remote job.
- An interrupted run may leave the host job running; cancel it from the run record or the host API.
Progress and interrupt
The run panel follows the job: status, step, loss, validation loss, curve, log tail and an Interrupt button. Starting a run scrolls it into view. The panel repaints itself while the job is active, including remote jobs.
Leaving Training and coming back restores the form from the active run, or from the last draft when nothing is running, so the fields keep the values that started the run. A finished run is also saved under the job's name; Load latest brings it back after a restart.
Validation and exam splits
Set the validation split under New model → Checkpoints & splits (Validation split (%) and Held-out exam (%)), or on Improve (Validation split and Exam split). Both default to 10% and must stay below 100% together. The exam slice is excluded from training and validation and is scored after the run for the promotion verdict.
Val loss is evaluated at each log interval, so the line stays — until the first logged step. If the split produces no held-out documents, the run log says so and training continues without a validation curve; load the job to see the percentages it actually used.
Checkpoints and chat
Checkpoints are written under the job's export folder (Training/checkpoints-<name>). A pretrain checkpoint holds a complete model.smm; a fine-tune checkpoint holds adapters.smlora.
- Add to Models copies a finished model or checkpoint into the Models folder, so Chat, Work and the labs see it.
- Chat does that and opens a chat with the model selected.
- Send to host uploads the model to a Compute host and activates it there; then pick that host in Chat.
- Fine-tune checkpoints are adapters only and cannot load in Chat. Rerun the fine-tune with Output = merged (optionally resuming the checkpoint) to get a merged .smm.
- For a remote run, the final export downloads back to this PC; the step checkpoints stay on the host.
- Interrupting a run keeps the latest checkpoint. Resume from it instead of starting over; Start fresh ignores it.
The Models tab lists files in the Training and Models folders. Anything still in Training shows Add to Models; a model already in the Models folder only shows Chat.
Accelerator errors
| Message | Meaning |
|---|---|
| only LoRA adapters are trainable in M1 | The job is a pretrain. Use Improve/SFT for GPU, or let it fall back to CPU. |
| LayerNorm, only RMSNorm | The base is GPT-2/BERT-style. Use a LLaMA/Qwen base for GPU training. |
| No CUDA or OpenCL device | The container has no GPU access. Check --gpus all, the driver and the toolkit. |
| MoE | MoE training is not supported by the GPU engine. |
| Val split produced no documents | The corpus is too small for a 64-token window; training continues without a validation curve. |