joey-1

baselabs/joey-1

作者
baselabs
任务
多模态
下载量
633
点赞数
38
趋势分
38
许可证
mit
收录时间
2026/11/4

license: mit library_name: transformers base_model: Qwen/Qwen3.5-2B pipeline_tag: text-classification language:

  • en tags:
  • decision-model
  • classification
  • calibration
  • routing
  • guardrails
  • vision
  • multimodal
  • baseten

joey-1

joey-1 is a 2B parameter decision model post-trained from Qwen3.5-2B by Base Labs at Baseten. Provide the model a state (text, optionally with images) and a set of questions and it will return a calibrated probability for every option, answering all questions from one shared-prefix prefill. On tested benchmarks:

  • Best model under 4B on the Jev Decision Index 0.4 public split: 63.30, #37 of 230 models.
  • 62.9 on a replica of the Artificial Analysis Decision Index.

For tasks that need broader general knowledge, see the 9B sibling elephant-1 and the 27B mammoth-1.

🗒️ Model Details

ModelParametersDescription
Qwen3.5-2B2BGeneral-purpose base model
joey-12BPost-trained for single-pass, calibrated decisions
elephant-19BLarger sibling
mammoth-127BLargest sibling, highest accuracy
  • Inputs: text and images
  • Training: full fine-tune of the language model on text and image decisions; the vision encoder is frozen
  • Attention: block-bidirectional mask on full-attention layers.
  • Max sequence length: 16,384 tokens

🏃 How to use

Baseten Model APIs

joey-1 is available on Baseten Model APIs at $0.01 per 1M input tokens, with zero data retention on by default. The endpoint accepts text and image inputs.

Self-hosted

setup.sh installs the pinned serving runtime (OneJev + serve.patch). It needs one CUDA GPU and the pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel image.

hf download baselabs/joey-1 --local-dir joey-1 && bash joey-1/setup.sh
from qev.engine import DecisionEngine
from qev.schema import SystemOneRequest

engine = DecisionEngine("baselabs/joey-1", device="cuda:0")

questions = {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {"billing": "Charges, refunds, invoices",
                     "technical": "App or site faults",
                     "fraud": "Suspected unauthorised use"},
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["Can wait", "Today", "Blocking the customer now"],
    },
}
resp, _ = engine.decide(SystemOneRequest(
    state="I was charged twice this month, please refund one of them.", questions=questions))
print(resp.answers)
typecriteriaanswer fields
noul: yes or nooptionalnoul: P(yes)
choice: one of named options{name: description}choice, probabilities
score: ordered levelslist of levels, lowest firstscore, probabilities

Requests longer than the trained length limits are refused, not truncated.

📊 Performance

Jev Decision Index 0.4 (public split)

ModelDecision Index
elephant-172.24
joey-163.30
Cloudflare Clef61.71
Jev 1.1460.70
Perplexity Decider Flash v1.260.06

On the public split, joey-1 trails elephant-1 in every area and beats Jev 1.14 in four of five (language, tools & automation, retrieval & classification, arts & human taste).

Artificial Analysis Decision Index (our replica)

ModelScore
mammoth-165.9
elephant-165.2
d1-ultra65
joey-162.9

🔧 Training (all details in the technical report)

  • Data: 6.21M decision examples (Mix 8). 58.3% core decision skills, 16.1% safety and abstention, 16.1% judging and expert domains, 9.5% public decision data.
  • Objective: cross-entropy plus a Brier calibration term (λ = 1), with a trained "I don't know" option.
  • Recipe: 1 epoch (43,390 steps), peak LR 1e-5, AdamW (β 0.9/0.95, no weight decay), 3% warm-up, cosine decay, bf16.
  • Hardware: 2 nodes × 8 H100 (16 GPUs).

Citation

@article{baseten2026mammoth,
  author  = {Base Labs, Baseten},
  title   = {mammoth-1 and elephant-1: Frontier Decision Models},
  journal = {Base Labs},
  year    = {2026},
  note    = {https://labs.baseten.co/articles/mammoth-1-and-elephant-1-frontier-decision-models},
}

内容来源:Hugging Face 模型卡

在 Hugging Face 查看