AI without
the token bill.

Record your actual hardware

On Windows, open Task Manager → Performance and note system RAM, GPU name and dedicated GPU memory. On a Mac, check About This Mac for the chip and unified memory. Also check free disk space.

Dedicated VRAM and system RAM are separate on a typical PC. Mac unified memory is shared with the rest of the system.

Use CanIRun.ai as a starting point

The independent checker offers device-based model discovery for text, coding, images and video. Confirm its selected device and memory values match your machine, then explore its model and device comparisons.

A fit estimate is a shortlist, not a guaranteed speed or quality result. Open the model publisher’s own model card and your chosen app’s hardware requirements before downloading.

Model size is only part of memory use

Quantisation compresses model weights, with potential quality tradeoffs. Long context, additional users and runtime overhead need memory too. A model download that nearly fills your VRAM may still fail to load.

CPU or partial GPU execution can widen your options, but responsiveness depends on the device and workload. Start with a smaller model and a short prompt.

LM Studio

A starting option for a visual chat interface, model downloads and document questions on Windows, macOS or Linux. It also provides a local API.

Download the model and required runtime first. Its offline documentation distinguishes local chat from online model search and downloads.

Ollama

Consider this when you want a local model service for compatible apps or scripts. Check GPU compatibility before choosing a model.

Ollama also offers cloud features. For a local-only setup, its FAQ documents OLLAMA_NO_CLOUD=1 and a restart. Keep the service on its default loopback address for a single-machine trial.

Jan

A desktop alternative for local model chat. Review the current desktop guide and hardware support to see whether it fits your operating system.

Select a local model deliberately: connecting a remote provider makes that conversation a remote workflow.

Open WebUI

A self-hosted interface for model backends such as Ollama. Useful if you want a browser workspace and are comfortable maintaining another service.

The interface does not itself make every backend local. Check the configured model endpoint and document-processing services before adding private files.

llama.cpp

For people who want direct control of GGUF model execution, hardware backends and a local server. Expect more setup decisions than a desktop chat app.

Use the official build or release instructions for your platform. A compatible file format does not guarantee every model architecture will work.

ComfyUI

A separate route for visual generation workflows. Start with an official beginner workflow and its required models rather than a text-chat model.

Check requirements for that exact workflow. Image size, video length and workflow components affect the resources needed; text-model fit does not establish image or video fit.

Install, download, disconnect

1. Choose a desktop app and read its system requirements.
2. Download one supported instruction/chat model that leaves memory headroom. Record the exact model version and quantisation.
3. Load the model and disable remote providers, web tools and cloud features.
4. Disconnect from the internet and try a short, non-sensitive note.

Use LM Studio’s offline checklist for that app. An offline answer confirms this trial worked without a connection; it does not audit every extension or future setting.

A prompt you can check

Illustrative input: “Meeting moved to Thursday at 10:00. Mia will send the revised drawing on Wednesday. Budget approval is still pending.”

Extract only the actions, owners and dates stated in this note. Return a table. Use “not stated” for missing details. Do not infer a calendar date.

Check that Mia’s drawing action stays on Wednesday and that no owner or deadline is invented for budget approval. Then test a slightly longer note and record waiting time, memory use and mistakes.

If it is slow or fails to load

• Close memory-heavy apps and reduce context length.
• Try a smaller supported model or a different quantisation.
• Check the runtime’s GPU support and which processor it is using.
• Compare the output on your actual task before considering an upgrade.

For Ollama, ollama ps reports CPU/GPU placement. See its official FAQ.