fix: 修复本地llm服务不可用问题 + 统一模型缓存目录位置
Some checks failed
构建并部署 AI Agent 服务 / deploy (push) Has been cancelled
Some checks failed
构建并部署 AI Agent 服务 / deploy (push) Has been cancelled
- 修复 config.py 添加 LOCAL_MODEL_NAME 配置项 - 修复 chat_services.py 本地模型检测时API路径重复问题(/v1/models -> /models) - 更新 .gitignore,移除模型目录跟踪 - 统一模型缓存到 docker/models/fastembed_cache,避免重复 - 更新 Dockerfile,正确复制预下载的BM25模型缓存
This commit is contained in:
@@ -27,7 +27,8 @@ from app.config import (
|
||||
VLLM_BASE_URL,
|
||||
LLM_API_KEY,
|
||||
ZHIPUAI_API_KEY,
|
||||
DEEPSEEK_API_KEY
|
||||
DEEPSEEK_API_KEY,
|
||||
LOCAL_MODEL_NAME
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -38,9 +39,9 @@ class LocalVLLMChatProvider(BaseServiceProvider[BaseChatModel]):
|
||||
本地 VLLM 生成式大模型服务提供者
|
||||
"""
|
||||
|
||||
def __init__(self, model: str = "gemma-4-E4B-it"):
|
||||
def __init__(self, model: str = None):
|
||||
super().__init__("local_vllm_chat")
|
||||
self._model = model
|
||||
self._model = model or LOCAL_MODEL_NAME
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""
|
||||
@@ -80,7 +81,7 @@ class LocalVLLMChatProvider(BaseServiceProvider[BaseChatModel]):
|
||||
headers["Authorization"] = f"Bearer {LLM_API_KEY}"
|
||||
|
||||
try:
|
||||
response = client.get("/v1/models", headers=headers)
|
||||
response = client.get("/models", headers=headers)
|
||||
if response.status_code == 200:
|
||||
logger.info(f"本地 VLLM 服务可用: {self._model}")
|
||||
return True
|
||||
|
||||
Reference in New Issue
Block a user