Reviewed November 3, 2026. App requirements are current to that review; benchmark report dates appear in the tables.
A local large language model runs its inference on your phone, tablet or computer. You download the app, runtime and model files before using them offline. The model must fit in usable memory, and its speed depends on the model, quantization, context length and software as well as the chip.
This guide covers iPhone, iPad and Mac apps, practical RAM budgets and seven published device configurations. The charts show generated tokens per second for specified test workloads. Use them as reference points for those workloads when setting expectations; a different model or chat app needs its own measurement.
How much RAM does a local LLM need?
Budget for model weights + context/KV cache + runtime buffers + the operating system and other apps. Storage capacity determines whether the download fits on disk. Installed RAM determines the device’s memory capacity, but an iPhone or iPad app cannot use all of it. A 256GB iPad’s storage is not 256GB of RAM.
Quantization stores model values with fewer bits. An ideal four-bit weight takes half a byte, so a dense 8-billion-parameter model has a weight-only lower bound near 4GB. Actual formats include scales, metadata and some higher-precision tensors. They also need working memory after loading. The table separates this arithmetic from practical device budgets.
Scroll horizontally to see the full RAM table.
| Model size | Ideal weights only | Planning target |
|---|---|---|
| 1–2B | 0.5–1GB | Start with 6–8GB mobile devices or an 8GB Mac and a supported small model. |
| 3–4B | 1.5–2GB | 8GB mobile where the app’s model picker permits; 8–16GB Mac with short context. |
| 7–8B | 3.5–4GB | Prefer a 16GB or larger Mac. Confirm the app’s allocation before trying this class on a tablet. |
| 13–14B | 6.5–7GB | 32GB Mac gives more room for context and other applications. |
| 30–32B | 15–16GB | 48–64GB Mac is a planning target with additional headroom; check the actual runtime estimate. |
Weights use decimal GB and the calculation parameters × 4 bits ÷ 8. Planning targets are AppShifted recommendations for quantized dense models, not universal minimums or measured allocations. Jan’s Mac guide gives 8/16/32GB examples for approximately 3B/7B/13B models. LM Studio recommends 16GB and qualifies smaller-model use on 8GB Macs. Architecture, context, quantization and simultaneous requests change the budget.
There are exceptions for optimized small models. Private LLM’s device guidance includes older 3GB and 4GB phones for selected builds. Use that app’s device/model catalog rather than applying those examples to an arbitrary GGUF file.
A concrete memory example
The Llama 2 7B Q4_0 weights used in the Mac benchmark are about 3.53GiB. For that model’s attention architecture, a single 4,096-token context with an FP16 KV cache adds a calculated 2GiB. That totals about 5.53GiB before runtime buffers, the app and macOS. This explains why the download size alone is insufficient.
Cache illustration: 2 × 32 layers × 32 KV heads × 128 dimensions × 4,096 tokens × 2 bytes = 2,147,483,648 bytes. This is a calculation, not a measured app allocation. Hugging Face explains KV-cache storage and quantization. Models with grouped-query attention or different cache types have different requirements. GB and GiB use different unit scales.
Devices that run local models: token-speed charts
These device configurations completed the linked tests. RAM values describe the tested hardware, not the minimum RAM required for the model. Contributor measurements are reproduced below; AppShifted did not run these benchmarks. Neither RAM capacity alone nor the chart order establishes model quality.
iPhone and iPad: TinyLlama 1.1B Q4_0
Generated tokens / second
| Device / chip configuration | Installed RAM | Decode tokens/s (reported spread) | Test record / reported date |
|---|---|---|---|
| iPhone 15 Pro | 8 GB | 56.86 ± 0.17 | Primary result · 2023-12-17 |
| iPhone 16 Pro | 8 GB | 69.77 ± 0.12 | Primary result · 2025-02-02 |
| iPad Pro 11-inch (M4, 256GB) | 8 GB | 121.20 ± 0.08 | Primary result · 2024-05-20 |
Mac: Llama 2 7B Q4_0
Generated tokens / second
| Device / chip configuration | Installed RAM | Decode tokens/s (reported spread) | Test record / reported date |
|---|---|---|---|
| MacBook Air M5 (10 GPU cores) | 16 GB | 31.88 ± 0.03 | Primary result · 2026-08-30 |
| M1 Ultra (64 GPU cores) | 128 GB | 113.79 ± 0.06 | Primary result · 2026-08-28 |
| M5 Max (40 GPU cores) | 128 GB | 119.92 ± 0.17 | Primary result · 2026-08-25 |
| Mac Studio M5 Ultra (80 GPU cores) | 256 GB | 179.10 ± 2.45 | Primary result · 2026-09-30 |
Read the test conditions before comparing speeds
- Different workloads: The mobile panel uses a 1.1B model; the Mac panel uses a 7B model. Compare rates with the model size and quantization visible. Neither panel measures a 14B or 32B model.
- Mobile report limits: The reports are from 2023–2025. The iPhone 15 Pro author checked the published SwiftUI recipe at commit
0e18b2e; the other two rows omit an independent build stamp and exact checkpoint revision. Treat them as reference examples. - Mac report limits: The four reports use the same recorded build and model class. Thread counts and recorded OS versions differ; cooling is not fully documented. Some chassis, context settings and file hashes were not recorded.
- Decode versus prefill:
tg128measures a short 128-token decode test.pp512is a separate prompt-processing test, not a 512-token prompt prepended to this generation measurement. The benchmark documentation describes the test types. - Application overhead: These tests exclude model loading, text tokenization, output sampling and interface work. Long conversations, battery settings and sustained heat can change a real session. The table’s ± values reproduce the sources’ displayed spread, not a confidence interval.
What tokens per second means for a reply
A token is a piece of text, not a fixed number of words. As a simple calculation, a 300-token reply at 30 generated tokens/second takes about ten seconds of generation after prompt processing. Model loading and time to the first token add to the wait. Reasoning models may also generate tokens before the visible answer.
The charts provide evidence for specific model/device tests. They do not supply an app-wide expected rate for PocketPal, Ollama, LM Studio or MLX-LM. For your expected chat speed, measure the selected model in the chosen app with the context and output length you normally use. No rate is assigned here to an untested configuration.
Apps and runtimes for local LLMs
App compatibility and model fit are separate checks. The requirements below describe the software; the model and context still need to fit. Desktop download links lead to the official providers. MLX-LM is a developer tool rather than a phone app.
PocketPal AI: iPhone and iPad GGUF chat
PocketPal AI on the App Store supports downloading or importing compatible GGUF models and running local chat. Its current listing requires iOS/iPadOS 15.1 or later. Start with a small supported model and a short context; the reviewed documentation gives no single RAM minimum covering every model.
The official setup guide explains model loading and metrics. Obtain the model first, then test on-device chat without a connection. Remote-server mode and optional diagnostic or benchmark submissions have separate network behavior. The project page links the current app and features; check the store for optional purchases.
Private LLM: a device-specific model catalog
Private LLM on the App Store offers prepared local models and Siri/Shortcuts integration. It is a one-time paid app. The mobile listing requires iOS/iPadOS 17 and A12 Bionic or later; its Mac version requires macOS 14.
Use the device/model catalog to select a build for the hardware. The provider recommends more memory for larger models and discusses memory pressure in its FAQ. Download the selected files before going offline. Operating-system compatibility does not mean every catalog model can be loaded on that device.
LM Studio: graphical Mac chat and a local API
Download LM Studio for a graphical interface to compatible local models, document chat and a local server. On Mac it requires Apple Silicon and macOS 14 or later; Intel Macs are outside its current support. The provider recommends 16GB RAM, while qualifying 8GB use for smaller models and modest context.
Download the required model and inference runtime before disconnecting. The offline guide distinguishes local operation from catalog searches and downloads. If a phone connects to LM Studio on a Mac, inference runs on the Mac; the phone’s RAM is not hosting the model. Windows and Linux versions have their own requirements.
Ollama: local models through an app, command line or API
Download Ollama to run supported models locally. Its Mac requirements specify macOS 14 or later: M-series Macs support CPU/GPU execution, while Intel x86 Macs use CPU execution. No one app-wide RAM figure covers every model and context.
Check the loaded model’s processor placement and memory behavior. Larger contexts and parallel requests consume more memory. Ollama also offers cloud models, which run remotely. For local-only use, select a downloaded local model; the FAQ documents disabling cloud features with OLLAMA_NO_CLOUD=1 followed by an application restart.
Jan: desktop chat with local and cloud selections
Download Jan for open-source desktop chat and a local API. Current Mac local inference requires Apple Silicon and macOS 13.6 or later. The Mac installation guide lists model-size memory examples and at least 10GB free storage. That storage requirement is separate from RAM.
Choose the local provider and download a compatible model before going offline. Jan can also connect to separately configured cloud providers. Current Intel-Mac remote-provider access does not establish local-model support in the bundled engine. Follow the quickstart and confirm the selected provider before testing.
MLX-LM: Python and command-line generation on Apple Silicon
MLX-LM runs supported models on Apple Silicon Macs and provides quantization and generation tools. Underlying MLX requires macOS 14 or later; the current MLX-LM package requires native Python 3.11 or later. Check the installation requirements and current package configuration.
Download dependencies, tokenizer and model files before attempting offline generation. Peak memory changes with weights, context cache and prompt-processing settings. The project’s large-model wired-memory optimization requires macOS 15 or later. Model training has a different memory budget from the inference covered here.
Choose a model, then measure your own configuration
- Check the hardware: Confirm installed RAM, chip and operating system. For an iPad, check the exact storage configuration because RAM can differ within the same product generation.
- Start with a small supported model: Use the app’s own catalog or compatibility guidance. A file ending in GGUF or an MLX model name does not prove that every architecture and quantization is supported by the installed runtime.
- Reserve context memory: Start with a modest context and one conversation. Increase it only after checking peak memory. Compare settings using the same model and quantization.
- Record useful timings: Note model-loading time, time to first token, generated tokens/second and output length separately. Repeat a short test and a longer session so initial speed is not mistaken for sustained performance.
- Verify local operation: Finish downloading, select local inference and test without a network connection. External tools, model searches and cloud providers can still need a connection.
On Mac, use Activity Monitor’s memory-pressure view alongside the runtime’s measurements. If memory pressure rises or loading fails, reduce the model or context before trying a larger workload. On mobile, follow the app’s model picker and watch for memory-related reloads or interrupted generation.
Sources: linked app providers, original benchmark contributors, pinned llama.cpp documentation and the cited memory references. Charts and memory-planning examples: AppShifted. This article covers inference, not training, and reports no AppShifted hardware tests. See also App Store categories and Apple devices.