streaming: add chat_with_tools_raw_streaming to LM Studio + LoggingChatbotClient
Nightly Build / build (push) Successful in 6m43s
Release / verify-version (pull_request) Successful in 2s
Release / release (pull_request) Has been skipped

LM Studio: delegate streaming to the inner OpenAI client.
LoggingChatbotClient: extract shared log_and_return helper, add
streaming override so deltas are forwarded and metadata still logged.
This commit is contained in:
2026-07-22 14:05:32 +01:00
parent 3343260bb0
commit 4dddc7d2ab
2 changed files with 78 additions and 36 deletions
+15 -1
View File
@@ -1,7 +1,8 @@
use async_trait::async_trait;
use serde_json::Value;
use tokio::sync::mpsc;
use crate::{ChatOptions, ChatResponse, ChatbotClient, LlmRawMeta, LlmTurn, Message, openai::OpenAiClient};
use crate::{ChatOptions, ChatResponse, ChatbotClient, LlmRawMeta, LlmTurn, Message, StreamDelta, openai::OpenAiClient};
/// LM Studio client.
///
@@ -48,4 +49,17 @@ impl ChatbotClient for LmStudioClient {
) -> anyhow::Result<(LlmTurn, Option<LlmRawMeta>)> {
self.inner.chat_with_tools_raw(messages, tools, options).await
}
/// LM Studio is OpenAI-compatible: streaming forwards to the inner client.
/// If a local build rejects `stream_options`, the inner pre-delta buffered
/// retry covers it transparently.
async fn chat_with_tools_raw_streaming(
&self,
messages: &[Value],
tools: &[Value],
options: &ChatOptions,
delta_tx: mpsc::Sender<StreamDelta>,
) -> anyhow::Result<(LlmTurn, Option<LlmRawMeta>)> {
self.inner.chat_with_tools_raw_streaming(messages, tools, options, delta_tx).await
}
}