Dolphin LLM Guide
The Dolphin family is a community-maintained collection of model variants and fine-tunes. Capabilities, licenses, and available weights differ by release; check the model card before downloading or deploying one.
Review model cards before use
“Uncensored” describes a model's training or fine-tuning claims, not a guarantee of accuracy, safety, or legal suitability. Review the specific model card, license, and intended-use guidance on Hugging Face before using a model with real data.
What is Dolphin?¶
Dolphin is a family of community-published language-model variants and fine-tunes. The maintainer, base model, training data, license, and intended use can differ by checkpoint, so the individual model card is more authoritative than a family name.
Key Characteristics¶
- Fine-tuned — Each checkpoint changes the behavior of a base model in a particular way
- Steerable — The runtime and system prompt influence the model's behavior
- Open weights — Access and licensing depend on the checkpoint and its base model
- Community maintained — Documentation and support are supplied by the publishing community
The Philosophy¶
Why Dolphin?
Local inference can give you more control over prompts, logs, and network access, but privacy is a deployment property—not a guarantee of the model family. Review the runtime, telemetry, API provider, and files you send to it.
Dolphin Model Family¶
Choosing a checkpoint¶
Start with the Dolphin 3.0 Llama 3.1 8B model card as a concrete example. For another checkpoint, compare its base model, license, context limit, quantization files, prompt format, and evaluation notes before downloading it.
Available Quantizations¶
Dolphin checkpoints may be published in quantized formats such as GGUF:
- Q4_K_M — Smaller download and lower memory use, with a quality tradeoff
- Q5_K_S — A larger quantized representation with a different quality/memory tradeoff
- Q8_0 — Higher precision and larger memory use
- EXL2 — A format used by compatible runtimes; follow that runtime's guidance
Technical jargon → In plain language
Technical jargon: Quantization changes how model weights are stored.
In plain language: It can make a model smaller and easier to run, but the choice may affect quality and supported runtimes.
Why it matters: Choose a file that your runtime supports, then test the responses on your own prompts before relying on it.
Deployment options¶
Choose a deployment that fits your constraints
Model weights may be available for local use, but running them still requires suitable hardware or an API provider. Check the specific model license and provider terms.
Common deployment paths¶
| Method | Requirements |
|---|---|
| Self-hosted (Ollama/LM Studio) | Hardware suitable for the selected model |
| Hugging Face Inference | A configured provider account |
| Cloud vLLM | A compatible hosted GPU environment |
Hardware planning¶
There is no universal VRAM table: requirements depend on parameter count, quantization, context length, runtime overhead, and whether the model is split across devices. Start with the model card and runtime estimate, leave headroom for the context window, and measure load time and generation speed on the machine that will run the workload.
Video Overview¶
How to Use¶
Using Ollama¶
# Pull a Dolphin model
ollama pull dolphin
# Or specific version
ollama pull dolphin3.0-llama3.1-8b
Using LM Studio¶
- Download LM Studio from lmstudio.ai
- Search for "dolphin" in the model browser
- Download your desired model
- Load and chat locally
Using Hugging Face Transformers¶
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "dphn/Dolphin3.0-Llama3.1-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
Using vLLM (Production)¶
vllm serve dphn/Dolphin3.0-Llama3.1-8B
Verification¶
- Last reviewed: 2026-09-09
- Primary sources: Dolphin 3.0 Llama 3.1 8B model card, Hugging Face model-loading documentation
- Scope: Model-card ownership, checkpoint naming, Transformers loading, licensing cautions, and deployment guidance were reviewed. Other Dolphin checkpoints and runtime commands require their own current model-card verification.
Resources¶
Disclaimer: You are responsible for evaluating model outputs, complying with the applicable model license, and keeping unsafe or sensitive workloads within an environment you control.