Tutorial 6 of 13 8 min

Setting Up Your Local LLM

Alistair's writing assistance runs on a language model that lives entirely on your machine. Nothing you write is ever sent anywhere. This walks through picking a model that fits your hardware, speeding it up with your graphics card, and using it as a writing coach and to interview your characters.

1

Find ML / LLM in Settings

Open Settings and go to ML / LLM under ML & Voice. This panel is your control center: it shows your detected hardware, your active model, and a shortcut into the Setup Wizard whenever you want to add another.

Settings modal, ML/LLM Interface panel, showing System Information (GPU, CPU, memory) and the active model configuration
2

Run the Setup Wizard

Click Download More Models to launch the Setup Wizard. It starts by scanning your own machine (processor, memory, and graphics card) and uses that to work out what size of model you can actually run well, instead of guessing.

Setup Wizard System Detection step, showing detected processor, memory, graphics card, and a recommended model size
3

Choose how to get a model

Continue to Model Selection. The quickest route is the Recommended model at the top, picked for your computer's memory: one click downloads it. Browse Hugging Face Models opens a curated, searchable library instead, and Import Local Model takes a .gguf file you already have, drag-and-drop. Downloads keep going if you close the window, and an interrupted one picks up where it stopped.

Setup Wizard Model Selection step, showing Browse Hugging Face Models and Import Local Model options
4

Pick a model sized to your hardware

The Model Browser opens with a handful of curated, writing-focused models up front, plus a search bar into thousands more GGUF models on Hugging Face, filtered to the size that suits your computer. Open a model to see its files, each labelled by what it trades off (Balanced, High quality, Best quality) with the memory it needs; the ones marked Recommended suit most computers.

Model Browser showing curated writing models: Llama 3.1 8B Instruct, Gemma 2 9B Instruct, and Hermes 3 Llama 3.1 8B, with a size filter and search bar
Bigger models give better answers but need more memory and run slower. If you're not sure, start with a 7B to 9B model: it's the sweet spot for most modern laptops. Alistair uses each model's own chat format automatically.
5

Use your graphics card

A graphics card runs models several times faster than the processor. Open Settings > Downloads: Alistair detects your card and offers the right download. NVIDIA cards get the GPU package (it also speeds up Text to Speech); AMD Radeon and Intel graphics get the Vulkan engine, about 20 MB. Restart Alistair after it installs.

Settings, Downloads panel, showing an AMD Radeon RX 7600 found and a Download GPU engine button for the Vulkan engine
No supported card? Models run on the processor. Any processor from about 2015 onward works (Intel 4th generation or AMD Ryzen and later).
6

Ask your writing coach

Open the chat from the top-right corner. Ask about the craft, talk through your world, or turn on Practice for a short exercise; paste what you write and it gives you feedback. Questions asks you pointed questions so you decide for yourself, and Reference cites your manuscript. When you want a draft, ask for one: Draft returns paste-ready prose and Dialogue returns spoken lines only. In Settings > Downloads, Keep model loaded sets how long the model stays in memory between uses.

The LLM chat with the Practice chip on, answering a question about showing grief with a short writing exercise
7

Interview your characters

In the Characters panel, click Interview on a character's card. They answer in their own voice from what you've written: their sheet, relationships, the lore they're part of, and the scenes they appear in (the passages used are shown under each answer). Pick a Focus (their past, how they talk, a relationship, a scene, or rehearsing a conversation where you play the other character) and the suggested questions follow it. Save to Notes keeps the interview, and the character stays consistent with it next time.

Character interview with Mira Alcott, focus set to her relationship with Tam Alcott, answering from the scene The Fire with the passage shown below
← Back to all tutorials