Setup guide

Getting started with MLX and mlx-lm on Apple Silicon

By Claudio A. G. M. Filho · Updated October 21, 2026

MLX is Apple's open-source array framework for machine learning on Apple Silicon, designed around unified memory so arrays don't need copying between CPU and GPU. mlx-lm is the companion package for running and fine-tuning language models.

Documentation-based setup guide. These commands have not been executed or benchmarked as part of this website review. Model downloads may be large; check the license, supported architecture, and available memory first.

Install

The reviewed mlx-lm 0.32.2 release requires Python 3.11 or newer. The example below uses python3.11; substitute your compatible interpreter if you use a newer version. If that command is missing, install a compatible version from Python's official macOS downloads before continuing. An older system Python is not sufficient.

python3.11 --version
python3.11 -m venv ~/macmodelfit-mlx && \
  source ~/macmodelfit-mlx/bin/activate && \
  python -m pip install "mlx-lm==0.32.2"

Use a new environment directory rather than overwriting an environment used by another project. The pin selects the reviewed MLX-LM version; dependencies and model revisions can still change. Review release notes before upgrading.

Generate text

The mlx-community organization on Hugging Face hosts thousands of pre-converted models.

mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --prompt "Explain unified memory in two sentences."

Chat and serve

mlx_lm.chat --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --host 127.0.0.1 --port 8080

mlx_lm.server exposes an OpenAI-compatible API for supported requests. Keep this development server on loopback; the project explicitly says it is not recommended for production. Client features and model support need individual checks.

Convert and quantize (optional, advanced)

Start with the pre-quantized model above for inference. Converting the original 7B checkpoint downloads roughly 14–15 GB of higher-precision weights, needs additional disk space, and can require much more memory than running its 4-bit conversion. A 16GB Mac may not have enough free memory for conversion. Check the original model’s current license and files first. This is an optional preparation step, not a requirement for using the pre-converted model.

mlx_lm.convert --hf-path mistralai/Mistral-7B-Instruct-v0.3 -q \
  --q-bits 4 --q-group-size 64 --mlx-path ./mistral-v03-mlx-4bit

This example explicitly requests affine 4-bit groups of 64 (approximately 4.5 effective bits per weight for tensors with 16-bit scales and biases — see our memory guide).

The output directory must not already exist. Choose another new --mlx-path if you repeat the conversion; the converter refuses to overwrite an existing directory.

Fine-tune with LoRA

LoRA trains adapter matrices rather than all model weights. It can require less memory than full fine-tuning, but inference estimates do not establish training fit. Follow the official MLX-LM training guide for the training dependencies, prepared JSONL dataset, model support and complete commands before starting.

Memory safety

Watch actual memory pressure while running a model. Runtime allocations and wired memory can reduce the room available to macOS. Prefer a smaller supported model or shorter context over changing system memory limits.