Provider System - Hyper-Extract.md3.2 KBit/ai/Provider System - Hyper-Extract.md
---
title: "Provider System - Hyper-Extract"
source_url: "https://yifanfeng97.github.io/Hyper-Extract/latest/concepts/provider-system/"
source_site: "yifanfeng97.github.io"
clipped_at: "2026-07-10T11:45:29.905398+00:00"
clipper: "aiwiki-url-ingest"
extractor: "readability_rendered"
source_strategy: "normal_web_clip"
source_strategy_label: "普通网页抓取"
---
# Provider System
Hyper\-Extract supports three ways to connect to LLMs: **OpenAI**, **Alibaba Bailian**, and **local vLLM**. All use the same `create_client()` interface — only the first line changes.
---
## Verified Model Compatibility
### Cloud API
> **Bailian users**: Both qwen\-max and deepseek\-v3 do not support `json_schema`. If you hit `messages must contain the word 'json'` or get non\-JSON output, switch to qwen\-plus, qwen\-turbo, or deepseek\-r1\. See [Issue \#24](https://github.com/yifanfeng97/Hyper-Extract/issues/24).
### Local Deployment
### Cloud Embedding
---
## Quick Start by Platform
```
from hyperextract import create_client
# OpenAI
llm, emb = create_client("openai", api_key="sk-xxx")
# Bailian
llm, emb = create_client("bailian", api_key="sk-xxx")
# Anthropic (Claude) — LLM only; Anthropic has no embeddings API, so
# pair it with an OpenAI-compatible embedder.
# Keys: ANTHROPIC_API_KEY (or CLAUDE_API_KEY) for the LLM, OPENAI_API_KEY for embeddings.
llm, emb = create_client(
llm="anthropic", # default model: claude-opus-4-8 (override with "anthropic:<model>")
embedder="openai:text-embedding-3-small",
)
# Local vLLM
llm, emb = create_client(
llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
embedder="vllm:bge-m3@http://localhost:8001/v1",
api_key="dummy",
)
```
Full configuration options: [Provider Configuration Guide](../../python/guides/provider-configuration/).
---
## vLLM Deployment
### Start LLM Service
```
vllm serve /path/to/qwen3.5-9b-gptq-marlin \
--served-model-name Qwen/Qwen3.5-9B \
--trust-remote-code \
--quantization gptq_marlin \
--dtype bfloat16 \
--max-model-len 8192 \
--gpu-memory-utilization 0.90 \
--default-chat-template-kwargs '{"enable_thinking": false}' \
--port 8000 \
--api-key dummy
```
### Start Embedding Service
```
vllm serve BAAI/bge-m3 \
--task embed \
--dtype float16 \
--max-model-len 8192 \
--port 8001
```
### Docker
```
docker run --runtime nvidia --gpus all \
--ipc=host \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 8000:8000 \
vllm/vllm-openai:latest \
--model Qwen/Qwen3.5-9B \
--trust-remote-code
```
---
## Recommendations
**Use non\-thinking models for local vLLM deployment.**
When deploying locally, thinking models (e.g. Qwen3\.5 thinking mode) output `<think>...</think>` tags, which conflict with constrained decoding's requirement for JSON from the first token. Disable thinking mode when deploying Qwen3\.5\-9B:
```
--default-chat-template-kwargs '{"enable_thinking": false}'
```
**DeepSeek\-R1 via Bailian is verified to work** — Bailian filters out `<think>` tags on the backend, so AutoGraph and `with_structured_output` work correctly. If accessing DeepSeek\-R1 through other channels, thinking output may still cause issues.
**Prefer GPTQ\-Marlin over AWQ** for quantization — AWQ has compatibility issues with vLLM 0\.21\.0\.
---
## References