CVE-2026-48148
Description
Budibase is an open-source low-code platform. Prior to 3.35.3, the VectorDB configuration endpoint in Budibase accepts a host parameter that undergoes no validation against internal IP ranges, reserved hostnames, or URL schemes. Any authenticated user with builder-level access can supply an arbitrary host value such as 169.254.169.254 or localhost, causing the server to initiate outbound TCP connections to internal network addresses or cloud metadata endpoints on their behalf.This vulnerability is fixed in 3.35.3.
AI Insight
LLM-synthesized narrative grounded in this CVE's description and references.
An unvalidated host parameter in Budibase VectorDB endpoint allows authenticated builders to perform SSRF attacks against internal networks.
Vulnerability
The VectorDB configuration endpoint in Budibase prior to 3.35.3 accepts a host parameter without validation against internal IP ranges, reserved hostnames, or URL schemes. The field is defined as Joi.string().required(), allowing any non-empty string to be passed directly to the database SDK for connection establishment [1].
Exploitation
An authenticated user with builder-level access can supply an arbitrary host value such as 169.254.169.254 or localhost. The server initiates TCP connections to the supplied address, and differences in connection timing and error messages allow the attacker to enumerate internal services and determine reachability [1]. A full proof-of-concept is provided in the advisory [1].
Impact
By exploiting this vulnerability, an attacker can probe internal network addresses and access cloud metadata endpoints (e.g., AWS EC2 metadata service at 169.254.169.254 or GCP metadata server at metadata.google.internal), leading to information disclosure of sensitive cloud instance metadata [1].
Mitigation
The vulnerability is fixed in Budibase version 3.35.3. Users should upgrade to this version or later. No workarounds are available [1].
AI Insight generated on May 27, 2026. Synthesized from this CVE's description and the cited reference URLs; citations are validated against the source bundle.
Affected products
2Patches
43a406ab0619dSimplify, remove vectordb and embedding models
36 files changed · +12 −2602
packages/backend-core/src/docIds/ids.ts+0 −4 modified@@ -163,10 +163,6 @@ export const generateWorkspaceFavouriteID = () => { return `${DocumentType.WORKSPACE_FAVOURITE}${SEPARATOR}${newid()}` } -export const generateVectorDbID = () => { - return `${DocumentType.VECTOR_STORE}${SEPARATOR}${newid()}` -} - export const generateKnowledgeBaseID = () => { return `${DocumentType.KNOWLEDGE_BASE}${SEPARATOR}${newid()}` }
packages/builder/src/pages/builder/workspace/[application]/agent/[agentId]/config.svelte+1 −5 modified@@ -89,11 +89,7 @@ Any constraints the agent must follow. let getCaretPosition: CaretPositionFn | undefined = $state.raw() let currentAgent: Agent | undefined = $derived($selectedAgent) - let completionConfigs = $derived( - ($aiConfigsStore.customConfigs || []).filter( - config => config.configType !== AIConfigType.EMBEDDINGS - ) - ) + let completionConfigs = $derived($aiConfigsStore.customConfigs || []) let modelOptions = $derived( completionConfigs.map(config => ({ label: config.name || config._id || "Unnamed",
packages/builder/src/settings/pages/ai/ProviderModelFields.svelte+2 −2 modified@@ -2,10 +2,10 @@ import { createEventDispatcher } from "svelte" import { Combobox, Select } from "@budibase/bbui" - import type { LLMProvider } from "@budibase/types" + import { type AIConfigType, type LLMProvider } from "@budibase/types" interface Props { - configType: "completions" | "embeddings" + configType: AIConfigType provider: string model: string providers?: LLMProvider[]
packages/builder/src/stores/portal/aiConfigs.ts+0 −1 modified@@ -35,7 +35,6 @@ export class AIConfigStore extends DerivedBudiStore< }, { [AIConfigType.COMPLETIONS]: [], - [AIConfigType.EMBEDDINGS]: [], } ), }))
packages/server/scripts/.env.ai-evals.example+0 −31 removed@@ -1,31 +0,0 @@ -# Base Budibase instance -BUDIBASE_BASE_URL=http://localhost:10000 -BUDIBASE_USERNAME=local@budibase.com -BUDIBASE_PASSWORD=cheekychuckles - -# Optional: pin a specific app. If omitted, the first dev app is used. -# BUDIBASE_APP_ID=app_dev_xxx - -# Optional: provider filter -# AI_EVAL_PROVIDERS=bbai,openai,azure-openai -# Note: OpenAI and Azure OpenAI evals run only in self mode. - -# Optional: Budibase mode filter -# AI_EVAL_BUDIBASE_MODES=self,cloud - -# Optional: global feature filter for all enabled providers. -# Allowed tokens: -# tables,js,cron,translate,classify,prompt,summarise,generate,extract,openai,all -# AI_EVAL_FEATURES=all - -# OpenAI -OPENAI_API_KEY= -OPENAI_MODEL=gpt-5-mini - -# Azure OpenAI -AZURE_OPENAI_API_KEY= -AZURE_OPENAI_BASE_URL=https://<resource>.openai.azure.com -AZURE_OPENAI_MODEL=gpt-4.1 - -# Budibase AI model override (optional) -BBAI_MODEL=gpt-5-mini
packages/server/scripts/rag-evals/.env.example+0 −32 removed@@ -1,32 +0,0 @@ -BUDIBASE_BASE_URL=http://localhost:10000 -BUDIBASE_USERNAME=local@budibase.com -BUDIBASE_PASSWORD=cheekychuckles -RAG_EVAL_APP_ID=app_dev_1579e422b54d440d8e6f4da9c2d90fb4 - -# Display name expected by /api/configs provider field -RAG_EVAL_PROVIDER=OpenAI -OPENAI_API_KEY=replace_me -OPENAI_BASE_URL=https://api.openai.com/v1 - -RAG_EVAL_VECTORDB_HOST=localhost -RAG_EVAL_VECTORDB_PORT=5432 -RAG_EVAL_VECTORDB_DATABASE=budibase -RAG_EVAL_VECTORDB_USER=bb_user -RAG_EVAL_VECTORDB_PASSWORD=secret -RAG_EVAL_COMPLETION_CONFIG_NAME=RAG Eval Completion -RAG_EVAL_EMBEDDING_CONFIG_NAME=RAG Eval Embedding -RAG_EVAL_VECTORDB_NAME=RAG Eval Vector DB -RAG_EVAL_KB_NAME=RAG Eval Knowledge Base -RAG_EVAL_AGENT_NAME=RAG Eval Agent - -# Optional overrides -# RAG_EVAL_FILE=rag-evals.json -# RAG_EVAL_KEEP_RESOURCES=1 -# RAG_EVAL_ENV_FILE=/absolute/path/to/.env.rag-evals -# RAG_EVAL_RAGAS_DOCKER_IMAGE=ragas-app -# RAG_EVAL_RAGAS_THRESHOLD=0.65 -# RAG_EVAL_RAGAS_MIN_CONTEXT_PRECISION=0.75 -# RAG_EVAL_RAGAS_MIN_CONTEXT_RECALL=0.90 -# RAG_EVAL_RAGAS_MIN_FAITHFULNESS=0.75 -# RAG_EVAL_RAGAS_MIN_ANSWER_RELEVANCY=0.70 -# RAG_EVAL_RAGAS_MIN_ANSWER_CORRECTNESS=0.70
packages/server/scripts/rag-evals/files/customer-support-operations-handbook.md+0 −64 removed@@ -1,64 +0,0 @@ -# Customer Support Operations Handbook - -## Overview - -- The Customer Support team provides assistance for onboarding, billing, and technical troubleshooting. -- Primary support hours are **08:00 to 20:00 UTC**, Monday through Friday. -- High-severity incidents are handled through a 24/7 on-call rotation. -- The official support mailbox is **support@example.com**. -- The internal queue identifier for urgent cases is **SUPPORT-P1**. - -## Account Management - -### Password Reset Policy -- Users can reset passwords from the account security page. -- Password reset links expire after **30 minutes**. -- Accounts are locked after **5 failed login attempts**. -- Locked accounts are automatically unlocked after **15 minutes**. - -### Access Requests -- Role changes require approval from a workspace administrator. -- Privileged role requests must include a business justification. -- Temporary elevated access expires after **72 hours** unless renewed. - -## Billing and Subscriptions - -### Invoices -- Standard billing cycle is monthly. -- Invoices are generated on the **1st day of each month**. -- Payment terms are **Net 30**. -- Billing questions should be routed to **billing@example.com**. - -### Plan Changes -- Upgrades take effect immediately. -- Downgrades take effect at the next renewal date. -- Prorated charges are calculated automatically for mid-cycle upgrades. - -## Incident Response - -### Severity Levels -- **SEV-1**: Full service outage affecting multiple customers. -- **SEV-2**: Major feature degradation with partial workaround. -- **SEV-3**: Minor issue with low business impact. - -### Response Targets -- First response target for SEV-1 is **15 minutes**. -- First response target for SEV-2 is **1 hour**. -- First response target for SEV-3 is **1 business day**. - -## Regional Offices - -| Location | Description | -|---|---| -| Paris | European support and compliance operations | -| New York | North American customer success operations | -| Singapore | Asia-Pacific escalations and after-hours coordination | - -## Contact Directory - -| Team | Contact | Purpose | -|---|---|---| -| Support Operations | support-ops@example.com | Queue triage and staffing | -| Security Operations | security@example.com | Security incidents and abuse reports | -| Billing | billing@example.com | Invoices, refunds, and payment methods | -| Customer Success | success@example.com | Adoption, onboarding, and renewals |
packages/server/scripts/rag-evals/index.ts+0 −1231 removed@@ -1,1231 +0,0 @@ -import { AIConfigType } from "@budibase/types" -import { existsSync, readFileSync, writeFileSync } from "fs" -import { spawnSync } from "child_process" -import { createHash } from "crypto" -import { tmpdir } from "os" -import { basename, extname, isAbsolute, join, relative, resolve } from "path" -import * as dotenv from "dotenv" - -type SupportedEmbeddingModel = "text-embedding-3-small" - -interface EvalExpectation { - containsAny?: string[] - containsAll?: string[] - sourceIncludesAny?: string[] - expectNoContext?: boolean -} - -interface EvalCase { - id: string - question: string - reference?: string - expectation?: EvalExpectation -} - -interface EvalFile { - embedding?: { - model?: SupportedEmbeddingModel - } - documents: string[] - cases: EvalCase[] -} - -interface RagasSample { - caseId: string - question: string - answer: string - contexts: string[] - reference?: string - sourceHints: string[] -} - -interface RagasOutput { - aggregate?: Record<string, number> - byCase?: Array<Record<string, unknown>> - raw?: unknown -} - -interface CreatedResources { - completionConfigId?: string - completionConfigCreated?: boolean - embeddingConfigId?: string - embeddingConfigCreated?: boolean - vectorDbId?: string - vectorDbCreated?: boolean - knowledgeBaseId?: string - knowledgeBaseCreated?: boolean - agentId?: string - agentCreated?: boolean -} - -interface ChatConversationMessage { - role: string - parts?: Array<{ type: string; text?: string }> - metadata?: { - ragSources?: Array<{ filename?: string; sourceId?: string }> - } -} - -interface ChatConversation { - _id?: string - messages: ChatConversationMessage[] -} - -interface UploadedFile { - filename: string - status: "processing" | "ready" | "failed" - errorMessage?: string -} - -interface RuntimeEnv { - budibaseBaseUrl: string - appId: string - budibaseUsername: string - budibasePassword: string - provider: string - openAIKey: string - openAIBaseUrl: string - chatModel: string - vectorDbHost: string - vectorDbPort: number - vectorDbDatabase: string - vectorDbUser: string - vectorDbPassword: string - completionConfigName: string - embeddingConfigName: string - vectorDbName: string - knowledgeBaseName: string - agentName: string - keepResources: boolean - ragasDockerImage: string - ragasMinContextPrecision?: number - ragasMinContextRecall?: number - ragasMinFaithfulness?: number - ragasMinAnswerRelevancy?: number - ragasMinAnswerCorrectness?: number - ragasThreshold?: number -} - -function color(code: number, text: string) { - return `\x1b[${code}m${text}\x1b[0m` -} - -const green = (text: string) => color(32, text) -const red = (text: string) => color(31, text) -const yellow = (text: string) => color(33, text) -const cyan = (text: string) => color(36, text) - -function section(text: string) { - return `\n=== ${text} ===` -} - -function wait(ms: number) { - return new Promise(resolvePromise => setTimeout(resolvePromise, ms)) -} - -function toAbsolutePath(baseDir: string, candidate: string) { - if (isAbsolute(candidate)) { - return candidate - } - return resolve(baseDir, candidate) -} - -function loadEnvFile() { - const configured = process.env.RAG_EVAL_ENV_FILE - const envPath = configured - ? toAbsolutePath(process.cwd(), configured) - : resolve(__dirname, ".env") - - if (!existsSync(envPath)) { - if (configured) { - throw new Error(`RAG_EVAL_ENV_FILE not found: ${envPath}`) - } - return - } - - dotenv.config({ path: envPath }) - console.log(yellow(`Loaded env: ${envPath}`)) -} - -function getRequiredEnv(name: string) { - const value = process.env[name] - if (!value || value.trim().length === 0) { - throw new Error(`Missing required env var: ${name}`) - } - return value.trim() -} - -function getOptionalEnv(name: string) { - const value = process.env[name] - if (!value || value.trim().length === 0) { - return undefined - } - return value.trim() -} - -function parseOptionalNumber(name: string): number | undefined { - const value = getOptionalEnv(name) - if (!value) { - return undefined - } - const parsed = Number(value) - if (!Number.isFinite(parsed)) { - throw new Error(`Invalid ${name}: ${value}`) - } - return parsed -} - -function getRuntimeEnv(): RuntimeEnv { - const vectorDbPortRaw = getRequiredEnv("RAG_EVAL_VECTORDB_PORT") - const vectorDbPort = Number(vectorDbPortRaw) - if (!Number.isInteger(vectorDbPort) || vectorDbPort <= 0) { - throw new Error( - `Invalid RAG_EVAL_VECTORDB_PORT: ${vectorDbPortRaw}. Must be a positive integer.` - ) - } - - return { - budibaseBaseUrl: getRequiredEnv("BUDIBASE_BASE_URL"), - appId: getRequiredEnv("RAG_EVAL_APP_ID"), - budibaseUsername: getRequiredEnv("BUDIBASE_USERNAME"), - budibasePassword: getRequiredEnv("BUDIBASE_PASSWORD"), - provider: getRequiredEnv("RAG_EVAL_PROVIDER"), - openAIKey: getRequiredEnv("OPENAI_API_KEY"), - openAIBaseUrl: getRequiredEnv("OPENAI_BASE_URL"), - chatModel: getOptionalEnv("CHAT_MODEL") || "gpt-4o-mini", - - vectorDbHost: getRequiredEnv("RAG_EVAL_VECTORDB_HOST"), - vectorDbPort, - vectorDbDatabase: getRequiredEnv("RAG_EVAL_VECTORDB_DATABASE"), - vectorDbUser: getRequiredEnv("RAG_EVAL_VECTORDB_USER"), - vectorDbPassword: getRequiredEnv("RAG_EVAL_VECTORDB_PASSWORD"), - completionConfigName: - getOptionalEnv("RAG_EVAL_COMPLETION_CONFIG_NAME") || - "RAG Eval Completion", - embeddingConfigName: - getOptionalEnv("RAG_EVAL_EMBEDDING_CONFIG_NAME") || "RAG Eval Embedding", - vectorDbName: - getOptionalEnv("RAG_EVAL_VECTORDB_NAME") || "RAG Eval Vector DB", - knowledgeBaseName: - getOptionalEnv("RAG_EVAL_KB_NAME") || "RAG Eval Knowledge Base", - agentName: getOptionalEnv("RAG_EVAL_AGENT_NAME") || "RAG Eval Agent", - keepResources: getOptionalEnv("RAG_EVAL_KEEP_RESOURCES") === "1", - ragasDockerImage: - getOptionalEnv("RAG_EVAL_RAGAS_DOCKER_IMAGE") || "ragas-app", - ragasMinContextPrecision: - parseOptionalNumber("RAG_EVAL_RAGAS_MIN_CONTEXT_PRECISION") || 0.75, - ragasMinContextRecall: - parseOptionalNumber("RAG_EVAL_RAGAS_MIN_CONTEXT_RECALL") || 0.9, - ragasMinFaithfulness: - parseOptionalNumber("RAG_EVAL_RAGAS_MIN_FAITHFULNESS") || 0.75, - ragasMinAnswerRelevancy: - parseOptionalNumber("RAG_EVAL_RAGAS_MIN_ANSWER_RELEVANCY") || 0.7, - ragasMinAnswerCorrectness: - parseOptionalNumber("RAG_EVAL_RAGAS_MIN_ANSWER_CORRECTNESS") || 0.7, - ragasThreshold: parseOptionalNumber("RAG_EVAL_RAGAS_THRESHOLD"), - } -} - -function parseEvalFile(evalFilePath: string): EvalFile { - if (!existsSync(evalFilePath)) { - throw new Error(`Eval file not found: ${evalFilePath}`) - } - const parsed = JSON.parse(readFileSync(evalFilePath, "utf-8")) as EvalFile - if (!Array.isArray(parsed.cases) || parsed.cases.length === 0) { - throw new Error("Eval file must include at least one case in 'cases'") - } - return parsed -} - -function getMimeType(filename: string) { - const ext = extname(filename).toLowerCase() - if (ext === ".pdf") { - return "application/pdf" - } - if (ext === ".md" || ext === ".markdown") { - return "text/markdown" - } - if (ext === ".yaml" || ext === ".yml") { - return "application/x-yaml" - } - return "text/plain" -} - -class ApiClient { - private baseUrl: string - private token = "" - private appId = "" - private csrfToken = "" - - constructor( - baseUrl: string, - private targetAppId: string, - private username: string, - private password: string - ) { - this.baseUrl = baseUrl.replace(/\/$/, "") - } - - async init() { - const loginResponse = await fetch( - `${this.baseUrl}/api/global/auth/default/login`, - { - method: "POST", - headers: { - "Content-Type": "application/json", - }, - body: JSON.stringify({ - username: this.username, - password: this.password, - }), - } - ) - if (!loginResponse.ok) { - const body = await loginResponse.text() - throw new Error(`Login failed (${loginResponse.status}): ${body}`) - } - - const token = - loginResponse.headers.get("x-budibase-token") || - loginResponse.headers.get("token") - if (!token) { - throw new Error("No auth token returned by login route") - } - this.token = token - - const apps = await this.request<Array<{ appId?: string; _id?: string }>>( - "GET", - "/api/applications?status=all", - undefined, - false - ) - const appIds = apps.map(app => app.appId || app._id).filter(Boolean) - if (!appIds.includes(this.targetAppId)) { - throw new Error( - `Configured app ${this.targetAppId} was not found. Available apps: ${appIds.join(", ")}` - ) - } - this.appId = this.targetAppId - - const self = await this.request<{ csrfToken?: string }>( - "GET", - "/api/self", - undefined, - true - ) - if (!self.csrfToken) { - throw new Error("Unable to fetch csrfToken from /api/self") - } - this.csrfToken = self.csrfToken - } - - clearSession() { - this.token = "" - this.csrfToken = "" - this.appId = "" - } - - async request<T>( - method: string, - path: string, - body?: unknown, - includeAppHeaders = true - ): Promise<T> { - const headers: Record<string, string> = { - "Content-Type": "application/json", - "x-budibase-token": this.token, - } - if (includeAppHeaders) { - headers["x-budibase-app-id"] = this.appId - } - if (method !== "GET") { - headers["x-csrf-token"] = this.csrfToken - } - - const response = await fetch(`${this.baseUrl}${path}`, { - method, - headers, - body: body ? JSON.stringify(body) : undefined, - }) - - const text = await response.text() - let payload: any = undefined - if (text) { - try { - payload = JSON.parse(text) - } catch { - payload = text - } - } - if (!response.ok) { - throw new Error( - `${method} ${path} failed (${response.status}): ${text || "<empty>"}` - ) - } - return payload as T - } - - async requestStream(path: string, body: unknown): Promise<void> { - const headers: Record<string, string> = { - "Content-Type": "application/json", - "x-budibase-token": this.token, - "x-budibase-app-id": this.appId, - "x-csrf-token": this.csrfToken, - } - const response = await fetch(`${this.baseUrl}${path}`, { - method: "POST", - headers, - body: JSON.stringify(body), - }) - if (!response.ok) { - const text = await response.text() - throw new Error( - `POST ${path} failed (${response.status}): ${text || "<empty>"}` - ) - } - if (!response.body) { - return - } - const reader = response.body.getReader() - while (true) { - const { done } = await reader.read() - if (done) { - break - } - } - } - - async uploadKnowledgeBaseFile( - knowledgeBaseId: string, - filename: string, - content: Buffer, - contentType: string - ): Promise<UploadedFile> { - const headers: Record<string, string> = { - "x-budibase-token": this.token, - "x-budibase-app-id": this.appId, - "x-csrf-token": this.csrfToken, - } - const formData = new FormData() - formData.append( - "file", - new Blob([content as any], { type: contentType }), - filename - ) - - const response = await fetch( - `${this.baseUrl}/api/knowledge-base/${knowledgeBaseId}/files`, - { - method: "POST", - headers, - body: formData, - } - ) - const text = await response.text() - let payload: any = undefined - if (text) { - try { - payload = JSON.parse(text) - } catch { - payload = text - } - } - if (!response.ok) { - throw new Error( - `POST /api/knowledge-base/${knowledgeBaseId}/files failed (${response.status}): ${text || "<empty>"}` - ) - } - if (!payload?.file) { - throw new Error(`Unexpected upload response: ${text || "<empty>"}`) - } - return payload.file as UploadedFile - } -} - -function extractAssistantText(chat: ChatConversation) { - const assistantMessages = (chat.messages || []).filter( - message => message.role === "assistant" - ) - const last = assistantMessages[assistantMessages.length - 1] - if (!last?.parts) { - return "" - } - return last.parts - .filter(part => part.type === "text" && part.text) - .map(part => part.text || "") - .join("") - .trim() -} - -function extractRagSourceHints(chat: ChatConversation) { - const assistantMessages = (chat.messages || []).filter( - message => message.role === "assistant" - ) - const last = assistantMessages[assistantMessages.length - 1] - const ragSources = last?.metadata?.ragSources || [] - return ragSources - .map(source => source.filename || source.sourceId || "") - .filter(Boolean) -} - -function normalizeText(value: string) { - return value.toLowerCase().replace(/\s+/g, " ").trim() -} - -function evaluateDeterministicExpectation( - answerText: string, - sourceHints: string[], - expectation?: EvalExpectation -) { - if (!expectation) { - return { passed: true, detail: "No expectation set" } - } - - const normalizedAnswer = normalizeText(answerText) - const normalizedSources = sourceHints.map(source => source.toLowerCase()) - const errors: string[] = [] - - if (expectation.expectNoContext && sourceHints.length > 0) { - errors.push( - `Expected no retrieved sources, but got: ${sourceHints.join(", ")}` - ) - } - - if (expectation.containsAny?.length) { - const anyMatch = expectation.containsAny.some(needle => - normalizedAnswer.includes(normalizeText(needle)) - ) - if (!anyMatch) { - errors.push( - `Missing any-of expected text: ${expectation.containsAny.join(" | ")}` - ) - } - } - - if (expectation.containsAll?.length) { - const missing = expectation.containsAll.filter( - needle => !normalizedAnswer.includes(normalizeText(needle)) - ) - if (missing.length > 0) { - errors.push(`Missing expected text: ${missing.join(", ")}`) - } - } - - if (expectation.sourceIncludesAny?.length) { - const anySource = expectation.sourceIncludesAny.some(needle => { - const normalizedNeedle = needle.toLowerCase() - return normalizedSources.some(source => source.includes(normalizedNeedle)) - }) - if (!anySource) { - errors.push( - `Missing expected source hint: ${expectation.sourceIncludesAny.join(" | ")}` - ) - } - } - - return { - passed: errors.length === 0, - detail: errors.length === 0 ? "OK" : errors.join("; "), - } -} - -function resolveReference(testCase: EvalCase) { - if (testCase.reference?.trim()) { - return testCase.reference.trim() - } - const containsAll = testCase.expectation?.containsAll || [] - if (containsAll.length > 0) { - return containsAll.join(". ") - } - const containsAny = testCase.expectation?.containsAny || [] - if (containsAny.length > 0) { - return containsAny[0] - } - return undefined -} - -async function waitForKnowledgeBaseFilesReady( - api: ApiClient, - knowledgeBaseId: string, - requiredFilenames: string[] -) { - const timeoutMs = 180_000 - const startedAt = Date.now() - while (Date.now() - startedAt < timeoutMs) { - const result = await api.request<{ files: UploadedFile[] }>( - "GET", - `/api/knowledge-base/${knowledgeBaseId}/files` - ) - const files = result.files || [] - const requiredStatuses = requiredFilenames.map(filename => { - const candidates = files.filter(file => file.filename === filename) - return { - filename, - ready: candidates.some(file => file.status === "ready"), - failed: candidates.filter(file => file.status === "failed"), - } - }) - - const readyCount = requiredStatuses.filter(status => status.ready).length - const failed = requiredStatuses.flatMap(status => - status.ready ? [] : status.failed - ) - - if (failed.length > 0) { - const details = failed - .map( - file => `${file.filename}: ${file.errorMessage || "unknown error"}` - ) - .join(", ") - throw new Error(`Knowledge base file ingestion failed: ${details}`) - } - - if (readyCount === requiredFilenames.length) { - return - } - - await wait(1500) - } - - throw new Error( - `Timed out waiting for knowledge base ingestion after ${timeoutMs}ms` - ) -} - -async function cleanupResources(api: ApiClient, created: CreatedResources) { - if (created.agentCreated && created.agentId) { - try { - await api.request("DELETE", `/api/agent/${created.agentId}`) - } catch (error) { - console.warn( - yellow(`Cleanup warning (agent): ${(error as any)?.message || error}`) - ) - } - } - if (created.knowledgeBaseCreated && created.knowledgeBaseId) { - try { - await api.request( - "DELETE", - `/api/knowledge-base/${created.knowledgeBaseId}` - ) - } catch (error) { - console.warn( - yellow( - `Cleanup warning (knowledge base): ${(error as any)?.message || error}` - ) - ) - } - } - if (created.vectorDbCreated && created.vectorDbId) { - try { - await api.request("DELETE", `/api/vectordb/${created.vectorDbId}`) - } catch (error) { - console.warn( - yellow( - `Cleanup warning (vector DB): ${(error as any)?.message || error}` - ) - ) - } - } - if (created.embeddingConfigCreated && created.embeddingConfigId) { - try { - await api.request("DELETE", `/api/configs/${created.embeddingConfigId}`) - } catch (error) { - console.warn( - yellow( - `Cleanup warning (embedding config): ${(error as any)?.message || error}` - ) - ) - } - } - if (created.completionConfigCreated && created.completionConfigId) { - try { - await api.request("DELETE", `/api/configs/${created.completionConfigId}`) - } catch (error) { - console.warn( - yellow( - `Cleanup warning (completion config): ${(error as any)?.message || error}` - ) - ) - } - } -} - -function shouldIgnoreAttachAgentError(error: unknown) { - const rawMessage = (error as any)?.message - const message = typeof rawMessage === "string" ? rawMessage.toLowerCase() : "" - return ( - message.includes("already") && - (message.includes("agent") || message.includes("exists")) - ) -} - -function findByName<T extends { name?: string }>(items: T[], name: string) { - const normalized = name.trim().toLowerCase() - return items.find(item => item.name?.trim().toLowerCase() === normalized) -} - -function withSettingsSuffix(baseName: string, suffix: string): string { - return `${baseName} ${suffix}` -} - -function mean(numbers: number[]) { - if (numbers.length === 0) { - return 0 - } - return numbers.reduce((acc, n) => acc + n, 0) / numbers.length -} - -function evaluateMetricRails( - runtimeEnv: RuntimeEnv, - aggregate: Record<string, number> -) { - const checks: Array<{ - metricName: string - min?: number - }> = [ - { - metricName: "context_precision", - min: runtimeEnv.ragasMinContextPrecision, - }, - { - metricName: "context_recall", - min: runtimeEnv.ragasMinContextRecall, - }, - { - metricName: "faithfulness", - min: runtimeEnv.ragasMinFaithfulness, - }, - { - metricName: "answer_relevancy", - min: runtimeEnv.ragasMinAnswerRelevancy, - }, - { - metricName: "answer_correctness", - min: runtimeEnv.ragasMinAnswerCorrectness, - }, - ] - - const failures: string[] = [] - for (const check of checks) { - if (typeof check.min !== "number") { - continue - } - const value = aggregate[check.metricName] - if (typeof value !== "number") { - failures.push( - `${check.metricName}: missing metric, expected >= ${check.min.toFixed(4)}` - ) - continue - } - if (value < check.min) { - failures.push( - `${check.metricName}: ${value.toFixed(4)} < ${check.min.toFixed(4)}` - ) - } - } - - return failures -} - -function runRagas(runtimeEnv: RuntimeEnv, samples: RagasSample[]): RagasOutput { - const outputDir = tmpdir() - - const runId = `${Date.now()}-${Math.random().toString(16).slice(2)}` - - const inputPath = join(outputDir, `ragas-input-${runId}.json`) - const outputPath = join(outputDir, `ragas-output-${runId}.json`) - - writeFileSync(inputPath, JSON.stringify({ samples }, null, 2), "utf-8") - console.log(cyan(`RAGAS input path: ${inputPath}`)) - console.log(cyan(`RAGAS output path: ${outputPath}`)) - - const executed = spawnSync( - "docker", - [ - "run", - "--rm", - "-e", - `OPENAI_API_KEY=${runtimeEnv.openAIKey}`, - "-e", - `OPENAI_BASE_URL=${runtimeEnv.openAIBaseUrl}`, - "-e", - `OPENAI_API_BASE=${runtimeEnv.openAIBaseUrl}`, - "-v", - `${outputDir}:/work`, - "-v", - `${__dirname}:/runner`, - runtimeEnv.ragasDockerImage, - "sh", - "-lc", - [ - `python /runner/ragas_runner.py /work/${basename(inputPath)} /work/${basename(outputPath)}`, - ].join(" && "), - ], - { - encoding: "utf-8", - } - ) - - if (executed.error) { - throw executed.error - } - if (executed.status !== 0) { - const stderr = executed.stderr?.trim() || "<empty>" - const stdout = executed.stdout?.trim() || "<empty>" - throw new Error( - `RAGAS runner failed with code ${executed.status}\nstdout:\n${stdout}\nstderr:\n${stderr}` - ) - } - - if (!existsSync(outputPath)) { - throw new Error("RAGAS runner did not create output file") - } - - return JSON.parse(readFileSync(outputPath, "utf-8")) as RagasOutput -} - -async function main() { - loadEnvFile() - const runtimeEnv = getRuntimeEnv() - - const dataDir = __dirname - const evalFilePath = toAbsolutePath( - dataDir, - process.env.RAG_EVAL_FILE || "rag-evals.json" - ) - const evalConfig = parseEvalFile(evalFilePath) - const embeddingModel = evalConfig.embedding?.model || "text-embedding-3-small" - - if (embeddingModel !== "text-embedding-3-small") { - throw new Error( - `Unsupported embedding model: ${embeddingModel}. Currently only text-embedding-3-small is supported.` - ) - } - - const chatModel = runtimeEnv.chatModel - - console.log(section("RAG API Evals (RAGAS)")) - console.log(cyan(`Eval file: ${evalFilePath}`)) - console.log(cyan(`Embedding model: ${embeddingModel}`)) - console.log(cyan(`Chat model: ${chatModel}`)) - - const documentPaths = evalConfig.documents.map(d => join(dataDir, d)) - if (documentPaths.length === 0) { - throw new Error(`No supported documents found in: ${dataDir}`) - } - console.log(cyan(`Documents: ${documentPaths.length}`)) - - const docTextByFilename = new Map<string, string>() - const docSignatures: string[] = [] - for (const path of documentPaths) { - const text = readFileSync(path).toString("utf-8") - const contentHash = createHash("sha256") - .update(text) - .digest("hex") - .slice(0, 16) - docSignatures.push(`${path}:${contentHash}`) - docTextByFilename.set(basename(path), text) - } - docSignatures.sort((a, b) => a.localeCompare(b)) - - const settingsSuffix = createHash("sha256") - .update( - JSON.stringify({ - provider: runtimeEnv.provider, - chatModel, - embeddingModel, - vectorDbHost: runtimeEnv.vectorDbHost, - vectorDbPort: runtimeEnv.vectorDbPort, - vectorDbDatabase: runtimeEnv.vectorDbDatabase, - vectorDbUser: runtimeEnv.vectorDbUser, - documents: docSignatures, - }) - ) - .digest("hex") - .slice(0, 12) - - const completionConfigName = withSettingsSuffix( - runtimeEnv.completionConfigName, - settingsSuffix - ) - const embeddingConfigName = withSettingsSuffix( - runtimeEnv.embeddingConfigName, - settingsSuffix - ) - const vectorDbName = withSettingsSuffix( - runtimeEnv.vectorDbName, - settingsSuffix - ) - const knowledgeBaseName = withSettingsSuffix( - runtimeEnv.knowledgeBaseName, - settingsSuffix - ) - const agentName = withSettingsSuffix(runtimeEnv.agentName, settingsSuffix) - - console.log(cyan(`Settings scope suffix: ${settingsSuffix}`)) - console.log( - cyan( - "Using mandatory unique resource names per provider/model/vector DB/doc set" - ) - ) - - const api = new ApiClient( - runtimeEnv.budibaseBaseUrl, - runtimeEnv.appId, - runtimeEnv.budibaseUsername, - runtimeEnv.budibasePassword - ) - await api.init() - - const created: CreatedResources = {} - try { - console.log(section("Provisioning")) - const configs = await api.request<any[]>("GET", "/api/configs") - - const existingCompletionConfig = findByName(configs, completionConfigName) - if (existingCompletionConfig?._id) { - created.completionConfigId = existingCompletionConfig._id - console.log(`Reusing completion config: ${completionConfigName}`) - } else { - const completionConfig = await api.request<any>("POST", "/api/configs", { - name: completionConfigName, - provider: runtimeEnv.provider, - model: chatModel, - credentialsFields: { - api_key: runtimeEnv.openAIKey, - api_base: runtimeEnv.openAIBaseUrl, - }, - configType: AIConfigType.COMPLETIONS, - }) - created.completionConfigId = completionConfig._id - created.completionConfigCreated = true - console.log(`Created completion config: ${completionConfigName}`) - } - - const existingEmbeddingConfig = findByName(configs, embeddingConfigName) - if (existingEmbeddingConfig?._id) { - created.embeddingConfigId = existingEmbeddingConfig._id - console.log(`Reusing embedding config: ${embeddingConfigName}`) - } else { - const embeddingConfig = await api.request<any>("POST", "/api/configs", { - name: embeddingConfigName, - provider: runtimeEnv.provider, - model: embeddingModel, - credentialsFields: { - api_key: runtimeEnv.openAIKey, - api_base: runtimeEnv.openAIBaseUrl, - }, - configType: AIConfigType.EMBEDDINGS, - }) - created.embeddingConfigId = embeddingConfig._id - created.embeddingConfigCreated = true - console.log(`Created embedding config: ${embeddingConfigName}`) - } - - const vectorDbs = await api.request<any[]>("GET", "/api/vectordb") - const existingVectorDb = findByName(vectorDbs, vectorDbName) - if (existingVectorDb?._id) { - created.vectorDbId = existingVectorDb._id - console.log(`Reusing vector DB: ${vectorDbName}`) - } else { - const vectorDb = await api.request<any>("POST", "/api/vectordb", { - name: vectorDbName, - provider: "pgvector", - host: runtimeEnv.vectorDbHost, - port: runtimeEnv.vectorDbPort, - database: runtimeEnv.vectorDbDatabase, - user: runtimeEnv.vectorDbUser, - password: runtimeEnv.vectorDbPassword, - }) - created.vectorDbId = vectorDb._id - created.vectorDbCreated = true - console.log(`Created vector DB: ${vectorDbName}`) - } - - const knowledgeBases = await api.request<any[]>( - "GET", - "/api/knowledge-base" - ) - const existingKnowledgeBase = findByName(knowledgeBases, knowledgeBaseName) - if (existingKnowledgeBase?._id) { - created.knowledgeBaseId = existingKnowledgeBase._id - console.log(`Reusing knowledge base: ${knowledgeBaseName}`) - } else { - const knowledgeBase = await api.request<any>( - "POST", - "/api/knowledge-base", - { - name: knowledgeBaseName, - embeddingModel: created.embeddingConfigId, - vectorDb: created.vectorDbId, - } - ) - created.knowledgeBaseId = knowledgeBase._id - created.knowledgeBaseCreated = true - console.log(`Created knowledge base: ${knowledgeBaseName}`) - } - - const agentsResponse = await api.request<{ agents: any[] }>( - "GET", - "/api/agent" - ) - const agents = agentsResponse.agents || [] - const existingAgent = findByName(agents, agentName) - if (existingAgent?._id) { - created.agentId = existingAgent._id - console.log(`Reusing agent: ${agentName}`) - } else { - const agent = await api.request<any>("POST", "/api/agent", { - name: agentName, - description: "RAG eval agent", - aiconfig: created.completionConfigId, - knowledgeBases: [created.knowledgeBaseId], - live: true, - }) - created.agentId = agent._id - created.agentCreated = true - console.log(`Created agent: ${agentName}`) - } - - const chatApp = await api.request<any>("GET", "/api/chatapps") - const chatAppId = chatApp?._id - if (!chatAppId) { - throw new Error("Could not resolve chat app ID") - } - - try { - await api.request("POST", `/api/chatapps/${chatAppId}/agent`, { - agentId: created.agentId, - }) - } catch (error) { - if (!shouldIgnoreAttachAgentError(error)) { - throw error - } - console.log( - yellow( - `Agent ${created.agentId} already attached to chat app ${chatAppId}, continuing` - ) - ) - } - - console.log(section("Uploading Docs")) - const existingFilesResponse = await api.request<{ files: UploadedFile[] }>( - "GET", - `/api/knowledge-base/${created.knowledgeBaseId!}/files` - ) - const existingFiles = existingFilesResponse.files || [] - const requiredFilenames: string[] = [] - - for (const docPath of documentPaths) { - const filename = relative(dataDir, docPath) || docPath - requiredFilenames.push(filename) - const alreadyPresent = existingFiles.some( - file => - file.filename === filename && - (file.status === "ready" || file.status === "processing") - ) - if (alreadyPresent) { - console.log(`Skipping upload (already present): ${filename}`) - continue - } - const fileBuffer = readFileSync(docPath) - await api.uploadKnowledgeBaseFile( - created.knowledgeBaseId!, - filename, - fileBuffer, - getMimeType(docPath) - ) - console.log(`Uploaded: ${filename}`) - } - - await waitForKnowledgeBaseFilesReady( - api, - created.knowledgeBaseId!, - requiredFilenames - ) - - console.log(section("Collecting Responses")) - const samples: RagasSample[] = [] - let guardrailTotal = 0 - let guardrailPassed = 0 - let guardrailFailed = false - - for (const testCase of evalConfig.cases) { - const createdConversation = await api.request<any>( - "POST", - `/api/chatapps/${chatAppId}/conversations`, - { - chatAppId, - agentId: created.agentId, - title: `RAGAS ${testCase.id}`, - } - ) - - const chatConversationId = createdConversation._id - await api.requestStream( - `/api/chatapps/${chatAppId}/conversations/${chatConversationId}/stream`, - { - _id: chatConversationId, - chatAppId, - agentId: created.agentId, - messages: [ - { - id: `${testCase.id}-user`, - role: "user", - parts: [{ type: "text", text: testCase.question }], - }, - ], - } - ) - - let conversation: ChatConversation | undefined - for (let attempt = 0; attempt < 8; attempt += 1) { - const fetched = await api.request<ChatConversation>( - "GET", - `/api/chatapps/${chatAppId}/conversations/${chatConversationId}` - ) - if ( - (fetched.messages || []).some(message => message.role === "assistant") - ) { - conversation = fetched - break - } - await wait(750) - } - - if (!conversation) { - throw new Error( - `Conversation ${chatConversationId} did not persist assistant response` - ) - } - - const answer = extractAssistantText(conversation) - const sourceHints = extractRagSourceHints(conversation) - const deterministicResult = evaluateDeterministicExpectation( - answer, - sourceHints, - testCase.expectation - ) - - if (testCase.expectation?.expectNoContext) { - guardrailTotal += 1 - if (deterministicResult.passed) { - guardrailPassed += 1 - console.log( - green(`GUARD PASS ${testCase.id}: ${deterministicResult.detail}`) - ) - } else { - guardrailFailed = true - console.log( - red(`GUARD FAIL ${testCase.id}: ${deterministicResult.detail}`) - ) - } - continue - } - - const contexts = sourceHints - .map( - hint => - docTextByFilename.get(hint) || docTextByFilename.get(basename(hint)) - ) - .filter((value): value is string => !!value) - - samples.push({ - caseId: testCase.id, - question: testCase.question, - answer, - contexts, - reference: resolveReference(testCase), - sourceHints, - }) - - console.log( - cyan( - `Collected ${testCase.id}: answer chars=${answer.length}, contexts=${contexts.length}` - ) - ) - } - - if (guardrailTotal > 0) { - console.log(section("No-Context Guardrails")) - console.log(`Total: ${guardrailTotal}`) - console.log(`Passed: ${guardrailPassed}`) - console.log(`Failed: ${guardrailTotal - guardrailPassed}`) - } - - let failed = guardrailFailed - - if (samples.length === 0) { - console.log( - yellow( - "Skipping RAGAS aggregate scoring because no informational RAG samples were collected" - ) - ) - } else { - console.log(section("RAGAS Scoring")) - const ragas = runRagas(runtimeEnv, samples) - const aggregate = ragas.aggregate || {} - const metricNames = Object.keys(aggregate).sort() - - if (metricNames.length === 0) { - throw new Error("RAGAS returned no aggregate metrics") - } - - for (const metricName of metricNames) { - const value = aggregate[metricName] - console.log(`${metricName}: ${value.toFixed(4)}`) - } - - const overall = mean(metricNames.map(name => aggregate[name])) - console.log(cyan(`overall_mean: ${overall.toFixed(4)}`)) - const railFailures = evaluateMetricRails(runtimeEnv, aggregate) - - if ( - typeof runtimeEnv.ragasThreshold === "number" && - overall < runtimeEnv.ragasThreshold - ) { - failed = true - console.log( - red( - `RAGAS overall mean ${overall.toFixed(4)} is below threshold ${runtimeEnv.ragasThreshold.toFixed(4)}` - ) - ) - } - - for (const failure of railFailures) { - failed = true - console.log(red(`RAGAS rail failed: ${failure}`)) - } - } - - if (failed) { - process.exitCode = 1 - } else { - console.log(green("RAGAS scoring completed")) - } - } finally { - if (!runtimeEnv.keepResources) { - await cleanupResources(api, created) - } else { - console.log( - yellow("Keeping resources because RAG_EVAL_KEEP_RESOURCES=1 is set") - ) - console.log(cyan(`Agent ID: ${created.agentId || "<none>"}`)) - console.log( - cyan(`Knowledge Base ID: ${created.knowledgeBaseId || "<none>"}`) - ) - console.log(cyan(`Vector DB ID: ${created.vectorDbId || "<none>"}`)) - } - api.clearSession() - } -} - -main().catch(error => { - const message = - typeof error === "string" - ? error - : (error as any)?.message || String(error || "Unknown error") - console.error(red(`RAGAS evals failed: ${message}`)) - process.exit(1) -})
packages/server/scripts/rag-evals/ragas.Dockerfile+0 −13 removed@@ -1,13 +0,0 @@ -FROM python:3.11-slim - -# Set working directory -WORKDIR /app - -# Install system deps (optional but useful) -RUN apt-get update && apt-get install -y build-essential - -# Install Python dependencies -RUN pip install ragas - -# Run your script -CMD ["python", "main.py"] \ No newline at end of file
packages/server/scripts/rag-evals/ragas_runner.py+0 −197 removed@@ -1,197 +0,0 @@ -#!/usr/bin/env python3 -import json -import math -import sys -from typing import Any, Dict, List, Tuple - - -def _import_deps(): - try: - from datasets import Dataset # type: ignore - from ragas import evaluate # type: ignore - from ragas.metrics import ( # type: ignore - answer_correctness, - answer_relevancy, - context_precision, - context_recall, - faithfulness, - ) - except Exception as err: - raise RuntimeError( - "Missing Python deps for RAGAS. Install with: " - "pip install ragas datasets openai" - ) from err - - return { - "Dataset": Dataset, - "evaluate": evaluate, - "metrics": { - "answer_correctness": answer_correctness, - "answer_relevancy": answer_relevancy, - "context_precision": context_precision, - "context_recall": context_recall, - "faithfulness": faithfulness, - }, - } - - -def _safe_float(value: Any) -> float: - try: - parsed = float(value) - except Exception: - return float("nan") - if math.isfinite(parsed): - return parsed - return float("nan") - - -def _mean(values: List[float]) -> float: - valid = [v for v in values if math.isfinite(v)] - if not valid: - return float("nan") - return sum(valid) / len(valid) - - -def _to_json_safe(value: Any) -> Any: - if isinstance(value, float): - if math.isfinite(value): - return value - return None - if isinstance(value, dict): - return {str(k): _to_json_safe(v) for k, v in value.items()} - if isinstance(value, list): - return [_to_json_safe(item) for item in value] - if isinstance(value, tuple): - return [_to_json_safe(item) for item in value] - if hasattr(value, "item"): - try: - return _to_json_safe(value.item()) - except Exception: - return str(value) - return value - - -def _to_rows(samples: List[Dict[str, Any]], schema: str) -> Dict[str, List[Any]]: - if schema == "legacy": - return { - "question": [row["question"] for row in samples], - "answer": [row["answer"] for row in samples], - "contexts": [row.get("contexts", []) for row in samples], - "ground_truth": [row.get("reference", "") or "" for row in samples], - "case_id": [row.get("caseId", "") for row in samples], - } - return { - "user_input": [row["question"] for row in samples], - "response": [row["answer"] for row in samples], - "retrieved_contexts": [row.get("contexts", []) for row in samples], - "reference": [row.get("reference", "") or "" for row in samples], - "case_id": [row.get("caseId", "") for row in samples], - } - - -def _evaluate( - evaluate: Any, Dataset: Any, metrics: Dict[str, Any], rows: Dict[str, List[Any]] -) -> Tuple[Dict[str, float], List[Dict[str, Any]], Dict[str, Any]]: - metric_list = [ - metrics["answer_correctness"], - metrics["answer_relevancy"], - metrics["context_precision"], - metrics["context_recall"], - metrics["faithfulness"], - ] - - dataset = Dataset.from_dict(rows) - result = evaluate(dataset=dataset, metrics=metric_list) - - pandas_rows: List[Dict[str, Any]] = [] - if hasattr(result, "to_pandas"): - try: - frame = result.to_pandas() - pandas_rows = frame.to_dict(orient="records") - except Exception: - pandas_rows = [] - - aggregates: Dict[str, float] = {} - metric_names = [ - "answer_correctness", - "answer_relevancy", - "context_precision", - "context_recall", - "faithfulness", - ] - - if pandas_rows: - for metric_name in metric_names: - values = [_safe_float(row.get(metric_name)) for row in pandas_rows] - metric_mean = _mean(values) - if math.isfinite(metric_mean): - aggregates[metric_name] = metric_mean - - if not aggregates: - maybe_dict = {} - if isinstance(result, dict): - maybe_dict = result - elif hasattr(result, "__dict__"): - maybe_dict = dict(getattr(result, "__dict__", {})) - - for metric_name in metric_names: - value = _safe_float(maybe_dict.get(metric_name)) - if math.isfinite(value): - aggregates[metric_name] = value - - return aggregates, pandas_rows, {"repr": repr(result)} - - -def main(): - if len(sys.argv) != 3: - raise RuntimeError("Usage: ragas_runner.py <input_json> <output_json>") - - input_path = sys.argv[1] - output_path = sys.argv[2] - - with open(input_path, "r", encoding="utf-8") as f: - payload = json.load(f) - - samples = payload.get("samples") - if not isinstance(samples, list) or len(samples) == 0: - raise RuntimeError("Input payload must include non-empty 'samples' array") - - deps = _import_deps() - Dataset = deps["Dataset"] - evaluate = deps["evaluate"] - metrics = deps["metrics"] - - errors = [] - for schema in ["legacy", "modern"]: - try: - rows = _to_rows(samples, schema) - aggregates, by_case, raw = _evaluate(evaluate, Dataset, metrics, rows) - if aggregates: - with open(output_path, "w", encoding="utf-8") as f: - json.dump( - _to_json_safe( - { - "schema": schema, - "aggregate": aggregates, - "byCase": by_case, - "raw": raw, - } - ), - f, - indent=2, - allow_nan=False, - ) - return - errors.append(f"Schema '{schema}' returned no aggregates") - except Exception as err: - errors.append(f"Schema '{schema}' failed: {err}") - - raise RuntimeError("RAGAS evaluation failed. " + " | ".join(errors)) - - -if __name__ == "__main__": - try: - main() - except Exception as err: - print(str(err), file=sys.stderr) - sys.exit(1)
packages/server/scripts/rag-evals/rag-evals.json+0 −70 removed@@ -1,70 +0,0 @@ -{ - "embedding": { - "provider": "openai", - "model": "text-embedding-3-small" - }, - "documents": ["files/customer-support-operations-handbook.md"], - "cases": [ - { - "id": "password-reset-expiry", - "question": "How long do password reset links remain valid?", - "reference": "30 minutes", - "expectation": { - "containsAny": ["30 minutes"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "failed-login-lockout", - "question": "After how many failed login attempts is an account locked?", - "reference": "5 failed login attempts", - "expectation": { - "containsAny": ["5 failed login attempts"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "billing-contact-email", - "question": "Which email should billing questions be sent to?", - "reference": "billing@example.com", - "expectation": { - "containsAny": ["billing@example.com"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "invoice-generation-date", - "question": "When are invoices generated?", - "reference": "the 1st day of each month", - "expectation": { - "containsAny": ["1st day of each month", "first day of each month"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "sev1-response-target", - "question": "What is the first response target for a SEV-1 incident?", - "reference": "15 minutes", - "expectation": { - "containsAny": ["15 minutes"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "urgent-queue-identifier", - "question": "What is the internal queue identifier for urgent support cases?", - "reference": "SUPPORT-P1", - "expectation": { - "containsAny": ["SUPPORT-P1"], - "sourceIncludesAny": ["customer-support-operations-handbook.md"] - } - }, - { - "id": "greeting-should-not-retrieve", - "question": "Good morning", - "expectation": { - "expectNoContext": true - } - } - ] -}
packages/server/src/api/routes/tests/ai/aiConfig.spec.ts+1 −215 modified@@ -1,31 +1,20 @@ import TestConfiguration from "../../../../tests/utilities/TestConfiguration" import nock from "nock" -import { db, docIds, encryption, features } from "@budibase/backend-core" +import { db, docIds, encryption } from "@budibase/backend-core" import { CustomAIProviderConfig, BUDIBASE_AI_PROVIDER_ID, PASSWORD_REPLACEMENT, WebSearchProvider, AIConfigType, CreateAIConfigRequest, - FeatureFlag, LiteLLMKeyConfig, } from "@budibase/types" import { context } from "@budibase/backend-core" import environment from "../../../../environment" import { licensing } from "@budibase/pro" import { mocks } from "@budibase/backend-core/tests" -jest.mock("../../../../sdk/workspace/ai/vectorDb/pgVectorDb", () => { - const actual = jest.requireActual( - "../../../../sdk/workspace/ai/vectorDb/pgVectorDb" - ) - return { - ...actual, - validatePgVectorDbConfig: jest.fn().mockResolvedValue(undefined), - } -}) - jest.mock("@budibase/pro", () => { const actual = jest.requireActual("@budibase/pro") return { @@ -70,10 +59,6 @@ const mockLiteLLMModelCostMap = () => .get("/public/litellm_model_cost_map") .reply(200, { "gpt-4o-mini": { litellm_provider: "openai", mode: "chat" }, - "text-embedding-3-small": { - litellm_provider: "openai", - mode: "embedding", - }, "claude-3-5-haiku": { litellm_provider: "anthropic", mode: "chat" }, "gpt-4o": { litellm_provider: ["openai", "azure"], mode: "responses" }, "groq/qwen/qwen3-32b": { litellm_provider: "groq", mode: "chat" }, @@ -147,15 +132,13 @@ describe("BudibaseAI", () => { expect(openAIProvider).toMatchObject({ models: { completions: ["gpt-4o", "gpt-4o-mini"], - embeddings: ["text-embedding-3-small"], }, }) const groqProvider = providers.find(provider => provider.id === "groq") expect(groqProvider).toMatchObject({ models: { completions: ["qwen/qwen3-32b"], - embeddings: [], }, }) }) @@ -817,203 +800,6 @@ describe("BudibaseAI", () => { }) }) - describe("embedding provider configs", () => { - const defaultEmbeddingRequest = { - name: "Embeddings Config", - provider: "OpenAI", - model: "text-embedding-3-large", - credentialsFields: { - api_key: "sk-test-key", - api_base: "https://api.openai.com", - }, - liteLLMModelId: "", - configType: AIConfigType.EMBEDDINGS, - } - - beforeEach(async () => { - await config.newTenant() - nock.cleanAll() - - mockLiteLLMProviders() - mockLiteLLMTeam() - }) - - it("creates an embedding config", async () => { - const embeddingValidationScope = nock(environment.LITELLM_URL) - .post("/v1/embeddings") - .reply(200, { data: [] }) - - const creationScope = nock(environment.LITELLM_URL) - .post("/key/generate") - .reply(200, { token_id: "embed-key-1", key: "embed-secret-1" }) - .post("/model/new") - .reply(200, { model_id: "embed-validation-1" }) - .post("/model/delete") - .reply(200, { status: "success" }) - .post("/model/new") - .reply(200, { model_id: "embed-model-1" }) - .post("/key/update") - .reply(200, { status: "success" }) - - const created = await config.api.ai.createConfig({ - ...defaultEmbeddingRequest, - }) - expect(created._id).toBeDefined() - expect(created.liteLLMModelId).toBe("embed-model-1") - expect(created.credentialsFields.api_key).toBe(PASSWORD_REPLACEMENT) - expect( - passwordMatch( - defaultEmbeddingRequest.credentialsFields.api_key, - (await getPersistedConfigAI(created._id)).credentialsFields.api_key - ) - ).toBeTrue() - - expect(creationScope.isDone()).toBe(true) - expect(embeddingValidationScope.isDone()).toBe(true) - - const configs = await config.api.ai.fetchConfigs() - expect( - configs.filter(c => c.configType === AIConfigType.EMBEDDINGS) - ).toHaveLength(1) - }) - - it("updates an embedding config", async () => { - const creationValidationScope = nock(environment.LITELLM_URL) - .post("/v1/embeddings") - .reply(200, { data: [] }) - - const creationScope = nock(environment.LITELLM_URL) - .post("/key/generate") - .reply(200, { token_id: "embed-key-2", key: "embed-secret-2" }) - .post("/model/new") - .reply(200, { model_id: "embed-validation-2" }) - .post("/model/delete") - .reply(200, { status: "success" }) - .post("/model/new") - .reply(200, { model_id: "embed-model-2" }) - .post("/key/update") - .reply(200, { status: "success" }) - - const created = await config.api.ai.createConfig({ - ...defaultEmbeddingRequest, - name: "Semantic Search", - }) - expect(creationScope.isDone()).toBe(true) - expect(creationValidationScope.isDone()).toBe(true) - - const updateValidationScope = nock(environment.LITELLM_URL) - .post("/v1/embeddings") - .reply(200, { data: [] }) - - const updateScope = nock(environment.LITELLM_URL) - .post("/model/new") - .reply(200, { model_id: "embed-validation-3" }) - .post("/model/delete") - .reply(200, { status: "success" }) - .patch(`/model/${created.liteLLMModelId}/update`) - .reply(200, { status: "success" }) - .post("/key/update") - .reply(200, { status: "success" }) - - const updated = await config.api.ai.updateConfig({ - ...created, - name: "Updated Embeddings", - model: "text-embedding-3-small", - }) - expect(updateScope.isDone()).toBe(true) - expect(updateValidationScope.isDone()).toBe(true) - expect(updated.name).toBe("Updated Embeddings") - }) - - it("deletes an embedding config and syncs LiteLLM models", async () => { - const creationValidationScope = nock(environment.LITELLM_URL) - .post("/v1/embeddings") - .reply(200, { data: [] }) - - const creationScope = nock(environment.LITELLM_URL) - .post("/key/generate") - .reply(200, { token_id: "embed-key-3", key: "embed-secret-3" }) - .post("/model/new") - .reply(200, { model_id: "embed-validation-4" }) - .post("/model/delete") - .reply(200, { status: "success" }) - .post("/model/new") - .reply(200, { model_id: "embed-model-3" }) - .post("/key/update") - .reply(200, { status: "success" }) - - const created = await config.api.ai.createConfig({ - ...defaultEmbeddingRequest, - }) - expect(creationScope.isDone()).toBe(true) - expect(creationValidationScope.isDone()).toBe(true) - - const deleteScope = nock(environment.LITELLM_URL) - .post("/key/update", body => { - expect(body).toMatchObject({ models: [] }) - return true - }) - .reply(200, { status: "success" }) - - const { deleted } = await config.api.ai.deleteConfig(created._id!) - expect(deleted).toBe(true) - expect(deleteScope.isDone()).toBe(true) - - const configsResponse = await config.api.ai.fetchConfigs() - expect( - configsResponse.filter(c => c.configType === AIConfigType.EMBEDDINGS) - ).toHaveLength(0) - }) - - it("allows deleting an embedding config when RAG is enabled", async () => { - await features.testutils.withFeatureFlags( - config.getTenantId(), - { [FeatureFlag.AI_RAG]: true }, - async () => { - const creationValidationScope = nock(environment.LITELLM_URL) - .post("/v1/embeddings") - .reply(200, { data: [] }) - - const creationScope = nock(environment.LITELLM_URL) - .post("/key/generate") - .reply(200, { token_id: "embed-key-4", key: "embed-secret-4" }) - .post("/model/new") - .reply(200, { model_id: "embed-validation-5" }) - .post("/model/delete") - .reply(200, { status: "success" }) - .post("/model/new") - .reply(200, { model_id: "embed-model-4" }) - .post("/key/update") - .reply(200, { status: "success" }) - - const created = await config.api.ai.createConfig({ - ...defaultEmbeddingRequest, - }) - expect(creationScope.isDone()).toBe(true) - expect(creationValidationScope.isDone()).toBe(true) - - const deleteScope = nock(environment.LITELLM_URL) - .post("/key/update", body => { - expect(body).toMatchObject({ models: [] }) - return true - }) - .reply(200, { status: "success" }) - - const { deleted } = await config.api.ai.deleteConfig(created._id!) - expect(deleted).toBe(true) - expect(deleteScope.isDone()).toBe(true) - - const configsResponse = await config.api.ai.fetchConfigs() - expect( - configsResponse.filter( - c => c.configType === AIConfigType.EMBEDDINGS - ) - ).toHaveLength(0) - } - ) - }) - }) - describe("workspace-specific LiteLLM key", () => { const defaultRequest: CreateAIConfigRequest = { name: "Test Config",
packages/server/src/automations/steps/ai/extract.spec.ts+0 −5 modified@@ -76,7 +76,6 @@ describe("extract file data step unit tests", () => { getDefaultLLMMock.mockResolvedValue({ chat: chatModel, - embedding: {} as any, providerOptions: undefined, uploadFile, }) @@ -135,7 +134,6 @@ describe("extract file data step unit tests", () => { getDefaultLLMMock.mockResolvedValue({ chat: chatModel, - embedding: {} as any, providerOptions: undefined, uploadFile: jest .fn() @@ -184,7 +182,6 @@ describe("extract file data step unit tests", () => { getDefaultLLMMock.mockResolvedValue({ chat: chatModel, - embedding: {} as any, uploadFile, providerOptions: undefined, }) @@ -225,7 +222,6 @@ describe("extract file data step unit tests", () => { getDefaultLLMMock.mockResolvedValue({ chat: createExtractMockLanguageModel([]), - embedding: {} as any, providerOptions: undefined, uploadFile: jest.fn().mockResolvedValue("file-123"), }) @@ -259,7 +255,6 @@ describe("extract file data step unit tests", () => { getDefaultLLMMock.mockResolvedValue({ chat: {} as any, - embedding: {} as any, providerOptions: undefined, uploadFile: jest.fn(), })
packages/server/src/sdk/workspace/ai/configs/index.ts+1 −14 modified@@ -219,7 +219,6 @@ export async function create( provider: config.provider, model: config.model, credentialFields: resolvedCredentialFields, - configType: config.configType, reasoningEffort: config.reasoningEffort, }) } else { @@ -365,7 +364,6 @@ export async function update( provider: updatedConfig.provider, name: updatedConfig.model, credentialFields: resolvedCredentialFields, - configType: updatedConfig.configType, reasoningEffort: updatedConfig.reasoningEffort, }) await liteLLM.syncKeyModels() @@ -441,7 +439,6 @@ export async function reconcileLiteLLMModels() { provider: existingConfig.provider, name: existingConfig.model, credentialFields: resolvedCredentialFields, - configType: existingConfig.configType, reasoningEffort: existingConfig.reasoningEffort, }) modelAlreadyExisted = true @@ -473,7 +470,6 @@ export async function reconcileLiteLLMModels() { provider: existingConfig.provider, model: existingConfig.model, credentialFields: resolvedCredentialFields, - configType: existingConfig.configType, reasoningEffort: existingConfig.reasoningEffort, }) console.log("Created LiteLLM model", { @@ -512,7 +508,6 @@ export async function fetchLiteLLMProviders(): Promise<LLMProvider[]> { liteLLMProviders = providers.map(provider => { const modelsByType = Object.entries(modelCostMap).reduce<{ completions: string[] - embeddings: string[] }>( (acc, [modelId, metadata]) => { const modelProvider = metadata?.litellm_provider @@ -543,10 +538,6 @@ export async function fetchLiteLLMProviders(): Promise<LLMProvider[]> { mode.trim().toLowerCase() ) - if (normalizedModes.includes("embedding")) { - acc.embeddings.push(normalizedModelId) - } - if ( !normalizedModes.length || normalizedModes.some(mode => @@ -558,16 +549,13 @@ export async function fetchLiteLLMProviders(): Promise<LLMProvider[]> { return acc }, - { completions: [], embeddings: [] } + { completions: [] } ) const models = { completions: [...new Set(modelsByType.completions)].sort((a, b) => a.localeCompare(b) ), - embeddings: [...new Set(modelsByType.embeddings)].sort((a, b) => - a.localeCompare(b) - ), } const mapProvider: RequiredKeys<LLMProvider> = { @@ -598,7 +586,6 @@ export async function fetchLiteLLMProviders(): Promise<LLMProvider[]> { externalProvider: "custom_openai", models: { completions: ["budibase/v1"], - embeddings: [], }, credentialFields: [ { key: "api_key", label: "api_key", field_type: "password" },
packages/server/src/sdk/workspace/ai/configs/litellm.ts+3 −70 modified@@ -5,7 +5,6 @@ import { locks, tenancy, } from "@budibase/backend-core" -import { utils } from "@budibase/shared-core" import { AIConfigType, BUDIBASE_AI_PROVIDER_ID, @@ -209,22 +208,19 @@ export async function addModel({ provider, model, credentialFields, - configType, reasoningEffort, }: { configId?: string provider: string model: string credentialFields: Record<string, string> - configType: AIConfigType reasoningEffort?: ReasoningEffort }): Promise<string> { configId ??= docIds.generateAIConfigID() const litellmParams = buildLiteLLMParams({ provider: await mapToLiteLLMProvider(provider), name: model, credentialFields, - configType, reasoningEffort, }) @@ -255,22 +251,19 @@ export async function updateModel({ provider, name, credentialFields, - configType, reasoningEffort, }: { configId: string llmModelId: string provider: string name: string credentialFields: Record<string, string> - configType: AIConfigType reasoningEffort?: ReasoningEffort }) { const litellmParams = buildLiteLLMParams({ provider: await mapToLiteLLMProvider(provider), name: name, credentialFields, - configType, reasoningEffort, }) @@ -318,62 +311,6 @@ export async function updateModel({ } } -async function validateEmbeddingConfig(model: { - provider: string - name: string - credentialFields: Record<string, string> -}) { - let modelId: string | undefined - - try { - modelId = await addModel({ - provider: model.provider, - model: model.name, - credentialFields: model.credentialFields, - configType: AIConfigType.EMBEDDINGS, - }) - - const response = await fetch(`${liteLLMUrl}/v1/embeddings`, { - method: "POST", - headers: { - "Content-Type": "application/json", - Authorization: liteLLMAuthorizationHeader, - }, - body: JSON.stringify({ - model: modelId, - input: "Budibase embedding validation", - }), - }) - - if (!response.ok) { - const text = await response.text() - throw new HTTPError(text || "Embedding validation failed", 500) - } - } catch (error) { - const message = error instanceof Error ? error.message : String(error) - throw new HTTPError(`Error validating configuration: ${message}`, 400) - } finally { - if (modelId) { - try { - await fetch(`${liteLLMUrl}/model/delete`, { - method: "POST", - headers: { - "Content-Type": "application/json", - Authorization: liteLLMAuthorizationHeader, - }, - body: JSON.stringify({ id: modelId }), - }) - } catch (e) { - console.error( - "Error deleting the temporary model for validating embeddings", - { e } - ) - } - } - } - return -} - async function validateCompletionsModel(model: { provider: string name: string @@ -432,14 +369,10 @@ export async function validateConfig(model: { credentialFields: Record<string, string> configType: AIConfigType }) { - switch (model.configType) { - case AIConfigType.EMBEDDINGS: - return validateEmbeddingConfig(model) - case AIConfigType.COMPLETIONS: - return validateCompletionsModel(model) - default: - throw utils.unreachable(model.configType) + if (model.configType !== AIConfigType.COMPLETIONS) { + throw new HTTPError(`Unsupported AI config type: ${model.configType}`, 400) } + return validateCompletionsModel(model) } export async function getKeySettings(): Promise<{
packages/server/src/sdk/workspace/ai/helpers/litellm.spec.ts+0 −16 modified@@ -1,12 +1,10 @@ -import { AIConfigType } from "@budibase/types" import { buildLiteLLMParams } from "./litellm" describe("buildLiteLLMParams", () => { const baseArgs = { provider: "openai", name: "gpt-5-mini", credentialFields: { api_key: "secret" }, - configType: AIConfigType.COMPLETIONS, } it("includes base params and credentials", () => { @@ -33,23 +31,11 @@ describe("buildLiteLLMParams", () => { expect(result).toMatchObject({ reasoning_effort: "low" }) }) - it("skips reasoning params for embeddings", () => { - const result = buildLiteLLMParams({ - ...baseArgs, - configType: AIConfigType.EMBEDDINGS, - reasoningEffort: "high", - }) - - expect(result).not.toHaveProperty("reasoning_effort") - expect(result).not.toHaveProperty("extra_body") - }) - it("normalizes groq qwen3-32b effort to default", () => { const result = buildLiteLLMParams({ provider: "groq", name: "qwen3-32b", credentialFields: {}, - configType: AIConfigType.COMPLETIONS, reasoningEffort: "high", }) @@ -61,7 +47,6 @@ describe("buildLiteLLMParams", () => { provider: "openrouter", name: "some-model", credentialFields: {}, - configType: AIConfigType.COMPLETIONS, reasoningEffort: "medium", }) @@ -76,7 +61,6 @@ describe("buildLiteLLMParams", () => { provider: "custom_openai", name: "some-model", credentialFields: {}, - configType: AIConfigType.COMPLETIONS, reasoningEffort: "medium", })
packages/server/src/sdk/workspace/ai/helpers/litellm.ts+1 −6 modified@@ -1,10 +1,9 @@ -import { AIConfigType, ReasoningEffort } from "@budibase/types" +import { ReasoningEffort } from "@budibase/types" export type BuildLiteLLMParamsArgs = { provider: string name: string credentialFields: Record<string, string> - configType: AIConfigType reasoningEffort?: ReasoningEffort } @@ -60,10 +59,6 @@ const applyCredentialParams: LiteLLMParamBuilder = (params, args) => ({ }) const applyReasoningParams: LiteLLMParamBuilder = (params, args) => { - if (args.configType === AIConfigType.EMBEDDINGS) { - return params - } - const normalizedReasoningEffort = normalizeReasoningEffort(args) if (!normalizedReasoningEffort) { return params
packages/server/src/sdk/workspace/ai/index.ts+0 −1 modified@@ -6,7 +6,6 @@ export * as deployments from "./deployments" export * as helpers from "./helpers" export * from "./instructions" export * as rag from "./rag" -export * as vectorDb from "./vectorDb" export * as knowledgeBase from "./knowledgeBase" export * as llm from "./llm" export * as agentLogs from "./agentLogs"
packages/server/src/sdk/workspace/ai/llm/bbai.ts+0 −3 modified@@ -204,9 +204,6 @@ export async function createBBAIClient( }) return { chat, - embedding: (() => { - throw new Error("BBAI embeddings are not supported") - }) as any, providerOptions: undefined, uploadFile: async ( stream: Readable,
packages/server/src/sdk/workspace/ai/llm/index.spec.ts+1 −1 modified@@ -121,7 +121,7 @@ describe("createLLM", () => { liteLLMModelId: "gpt-5-mini", } as any) - const expected = { chat: "chat", embedding: "embedding" } + const expected = { chat: "chat" } createLiteLLMOpenAIMock.mockResolvedValue(expected as any) const result = await createLLM("config-2", "session-1")
packages/server/src/sdk/workspace/ai/llm/litellm.ts+0 −1 modified@@ -54,7 +54,6 @@ export const createLiteLLMOpenAI = async ( const llm = createOpenAI(clientConfig) return { chat: llm.chat(modelId), - embedding: llm.embedding(modelId), providerOptions: getLiteLLMProviderOptions, uploadFile: async (stream: Readable, filename: string) => { const fileId = await uploadFile({
packages/server/src/sdk/workspace/ai/vectorDb/crud.ts+0 −113 removed@@ -1,113 +0,0 @@ -import { context, docIds, HTTPError } from "@budibase/backend-core" -import { - DocumentType, - PASSWORD_REPLACEMENT, - SEPARATOR, - VectorDbProvider, - VectorDb, -} from "@budibase/types" -import { resolvePgVectorDbConfig, validatePgVectorDbConfig } from "./pgVectorDb" -import { utils } from "@budibase/shared-core" - -const assertValidVectorDbId = (id: string) => { - const prefix = `${DocumentType.VECTOR_STORE}${SEPARATOR}` - if (!id?.startsWith(prefix)) { - throw new HTTPError("Invalid vector database id", 400) - } -} - -export async function fetch(): Promise<VectorDb[]> { - const db = context.getWorkspaceDB() - const result = await db.allDocs<VectorDb>( - docIds.getDocParams(DocumentType.VECTOR_STORE, undefined, { - include_docs: true, - }) - ) - - return result.rows.map(row => row.doc).filter((doc): doc is VectorDb => !!doc) -} - -export async function find(id: string): Promise<VectorDb | undefined> { - assertValidVectorDbId(id) - const db = context.getWorkspaceDB() - const result = await db.tryGet<VectorDb>(id) - if (!result || result._deleted) { - return undefined - } - return result -} - -export async function create(config: VectorDb): Promise<VectorDb> { - const db = context.getWorkspaceDB() - - const newConfig: VectorDb = { - _id: docIds.generateVectorDbID(), - name: config.name, - provider: config.provider, - host: config.host, - port: config.port, - database: config.database, - user: config.user, - password: config.password, - } - - switch (newConfig.provider) { - case VectorDbProvider.PGVECTOR: - await validatePgVectorDbConfig(await resolvePgVectorDbConfig(newConfig)) - break - default: - utils.unreachable(newConfig.provider, { doNotThrow: true }) - throw new HTTPError("Unsupported vector database provider", 400) - } - - const { rev } = await db.put(newConfig) - newConfig._rev = rev - - return newConfig -} - -export async function update(config: VectorDb): Promise<VectorDb> { - if (!config._id || !config._rev) { - throw new HTTPError("id and rev required", 400) - } - assertValidVectorDbId(config._id) - - const db = context.getWorkspaceDB() - const existing = await db.tryGet<VectorDb>(config._id) - if (!existing) { - throw new HTTPError("Vector store config not found", 404) - } - - const password = - config.password === PASSWORD_REPLACEMENT - ? existing.password - : config.password - - const updated: VectorDb = { - ...existing, - ...config, - password, - } - - switch (updated.provider) { - case VectorDbProvider.PGVECTOR: - await validatePgVectorDbConfig(await resolvePgVectorDbConfig(updated)) - break - default: - throw new HTTPError("Unsupported vector database provider", 400) - } - - const { rev } = await db.put(updated) - updated._rev = rev - - return updated -} - -export async function remove(id: string) { - assertValidVectorDbId(id) - - const db = context.getWorkspaceDB() - - const existing = await db.get<VectorDb>(id) - await db.remove(existing) -}
packages/server/src/sdk/workspace/ai/vectorDb/index.ts+0 −2 removed@@ -1,2 +0,0 @@ -export * from "./crud" -export * from "./utils"
packages/server/src/sdk/workspace/ai/vectorDb/pgVectorDb.spec.ts+0 −112 removed@@ -1,112 +0,0 @@ -const mockClient = { - connect: jest.fn(), - end: jest.fn(), - query: jest.fn(), -} - -jest.mock("pg", () => ({ - Client: jest.fn(() => mockClient), -})) - -jest.mock("@budibase/backend-core", () => { - const actual = jest.requireActual("@budibase/backend-core") - return { - ...actual, - context: { - ...actual.context, - getOrThrowWorkspaceId: jest.fn(() => "ws_dev_123"), - }, - db: { - ...actual.db, - getProdWorkspaceID: jest.fn((id: string) => id.replace("_dev", "")), - }, - tenancy: { - ...actual.tenancy, - getTenantId: jest.fn(() => "tenant_123"), - }, - } -}) - -import { HTTPError } from "@budibase/backend-core" -import { VectorDbProvider } from "@budibase/types" -import { buildPgVectorDbConfig, validatePgVectorDbConfig } from "./pgVectorDb" - -describe("pgVectorDb", () => { - beforeEach(() => { - jest.clearAllMocks() - }) - - describe("validatePgVectorDbConfig", () => { - const config = { - name: "test", - provider: VectorDbProvider.PGVECTOR, - host: "localhost", - port: 5432, - database: "test", - user: "user", - password: "pass", - } - - it("validates a working pgvector connection", async () => { - mockClient.query - .mockResolvedValueOnce({ rows: [] }) - .mockResolvedValueOnce({ rowCount: 1, rows: [{ "?column?": 1 }] }) - - await validatePgVectorDbConfig(config) - - expect(mockClient.connect).toHaveBeenCalledTimes(1) - expect(mockClient.query).toHaveBeenNthCalledWith(1, "SELECT 1") - expect(mockClient.query).toHaveBeenNthCalledWith( - 2, - "SELECT 1 FROM pg_extension WHERE extname = 'vector' LIMIT 1" - ) - expect(mockClient.end).toHaveBeenCalledTimes(1) - }) - - it("rejects databases without pgvector installed", async () => { - mockClient.query - .mockResolvedValueOnce({ rows: [] }) - .mockResolvedValueOnce({ rowCount: 0, rows: [] }) - - const result = validatePgVectorDbConfig(config) - await expect(result).rejects.toThrow(HTTPError) - await expect(result).rejects.toThrow("pgvector extension installed") - }) - - it("surfaces connection failures as validation errors", async () => { - mockClient.connect.mockRejectedValueOnce( - new Error("connect ECONNREFUSED") - ) - - await expect(validatePgVectorDbConfig(config)).rejects.toThrow( - "connect ECONNREFUSED" - ) - expect(mockClient.end).toHaveBeenCalledTimes(1) - }) - }) - - describe("deleteBySourceIds", () => { - it("does not throw when the table does not exist", async () => { - mockClient.query.mockRejectedValue({ code: "42P01" }) - - const vectorDb = buildPgVectorDbConfig( - { - name: "test", - provider: VectorDbProvider.PGVECTOR, - host: "localhost", - port: 5432, - database: "test", - user: "user", - password: "pass", - }, - { namespaceId: "agent_123" } - ) - - await vectorDb.deleteBySourceIds(["source_1"]) - - expect(mockClient.connect).toHaveBeenCalledTimes(1) - expect(mockClient.query).toHaveBeenCalledTimes(1) - expect(mockClient.end).toHaveBeenCalledTimes(1) - }) - }) -})
packages/server/src/sdk/workspace/ai/vectorDb/pgVectorDb.ts+0 −287 removed@@ -1,287 +0,0 @@ -import { VectorDbProvider, type VectorDb as VectorDbDoc } from "@budibase/types" -import * as crypto from "crypto" -import { Client } from "pg" -import { - context, - db as dbCore, - HTTPError, - tenancy, -} from "@budibase/backend-core" -import { - isEnvironmentVariableKey, - processEnvironmentVariable, -} from "../../../utils" -import type { - ChunkInput, - PgVectorDbConfig, - QueryResultRow, - VectorDb, -} from "./types" - -const vectorLiteral = (values: number[]) => - `[${values.map(value => Number(value) || 0).join(",")}]` - -const TABLE_PREFIX = "bb_chunks_" -const TABLE_HASH_LENGTH = 10 - -const buildAgentTableName = (namespaceId: string) => { - const normalized = namespaceId - .toLowerCase() - .replace(/[^a-z0-9]+/g, "_") - .replace(/^_+|_+$/g, "") - const currentWorkspaceId = context.getOrThrowWorkspaceId() - const prodWorkspaceId = dbCore.getProdWorkspaceID(currentWorkspaceId) - const hash = crypto - .createHash("sha256") - .update(`${tenancy.getTenantId()}:${prodWorkspaceId}:${namespaceId}`) - .digest("hex") - .slice(0, TABLE_HASH_LENGTH) - const maxBaseLength = 63 - TABLE_PREFIX.length - 1 - TABLE_HASH_LENGTH - const base = (normalized || "agent").slice(0, Math.max(0, maxBaseLength)) - return `${TABLE_PREFIX}${base}_${hash}` -} - -const buildPgConnectionString = (config: VectorDbDoc) => { - const userPart = config.user ? encodeURIComponent(config.user) : "" - const passwordPart = config.password - ? `:${encodeURIComponent(config.password)}` - : "" - const auth = userPart ? `${userPart}${passwordPart}@` : "" - return `postgresql://${auth}${config.host}:${config.port}/${config.database}` -} - -const resolvePort = async (portConfig: string | number): Promise<number> => { - if (typeof portConfig === "number") { - return portConfig - } - - if (!isEnvironmentVariableKey(portConfig)) { - return Number(portConfig) - } - - const envVariableValue = await processEnvironmentVariable(portConfig) - return Number(envVariableValue) -} - -export const resolvePgVectorDbConfig = async (config: VectorDbDoc) => { - return { - ...config, - host: await processEnvironmentVariable(config.host), - port: await resolvePort(config.port), - database: await processEnvironmentVariable(config.database), - user: config.user - ? await processEnvironmentVariable(config.user) - : config.user, - password: config.password - ? await processEnvironmentVariable(config.password) - : config.password, - } -} - -export const validatePgVectorDbConfig = async (config: VectorDbDoc) => { - const client = new Client({ - connectionString: buildPgConnectionString(config), - }) - - try { - await client.connect() - await client.query("SELECT 1") - const result = await client.query( - "SELECT 1 FROM pg_extension WHERE extname = 'vector' LIMIT 1" - ) - - if (!result.rowCount) { - throw new HTTPError( - "The target PostgreSQL database does not have the pgvector extension installed", - 400 - ) - } - } catch (err: any) { - if (err instanceof HTTPError) { - throw err - } - throw new HTTPError( - "Could not validate the configuration: " + - (err?.message || "Failed to connect to the vector database"), - 400 - ) - } finally { - await client.end() - } -} - -export const buildPgVectorDbConfig = ( - config: VectorDbDoc, - options: { namespaceId: string } -) => { - return new PgVectorDb({ - provider: VectorDbProvider.PGVECTOR, - databaseUrl: buildPgConnectionString(config), - tableName: buildAgentTableName(options.namespaceId), - }) -} - -class PgVectorDb implements VectorDb { - private readonly tableName: string - - constructor(private readonly config: PgVectorDbConfig) { - if (!/^[a-z0-9_]+$/.test(config.tableName)) { - throw new Error("Invalid vector table name") - } - this.tableName = config.tableName - } - - private async withClient<T>(handler: (client: Client) => Promise<T>) { - const client = new Client({ - connectionString: this.config.databaseUrl, - }) - await client.connect() - try { - return await handler(client) - } finally { - await client.end() - } - } - - private async ensureSchema(client: Client, embeddingDimensions: number) { - const buildIndexName = (tableName: string, suffix: string) => { - const hash = crypto - .createHash("sha256") - .update(`${tableName}:${suffix}`) - .digest("hex") - .slice(0, 20) - return `bb_sc_idx_${hash}` - } - - await client.query("CREATE EXTENSION IF NOT EXISTS vector") - await client.query(` - CREATE TABLE IF NOT EXISTS ${this.tableName} ( - id SERIAL PRIMARY KEY, - source TEXT NOT NULL, - chunk_hash TEXT NOT NULL, - chunk_text TEXT NOT NULL, - embedding vector(${embeddingDimensions}) NOT NULL, - created_at TIMESTAMPTZ DEFAULT NOW() - ) - `) - await client.query( - `CREATE UNIQUE INDEX IF NOT EXISTS ${buildIndexName(this.tableName, "source_chunk_hash_uq")} ON ${this.tableName} (source, chunk_hash)` - ) - await client.query(` - CREATE INDEX IF NOT EXISTS ${buildIndexName( - this.tableName, - "embedding" - )} - ON ${this.tableName} - USING ivfflat (embedding vector_cosine_ops) - WITH (lists = 100) - `) - } - - async upsertSourceChunks( - sourceId: string, - chunks: ChunkInput[] - ): Promise<{ inserted: number; total: number }> { - if (!chunks.length) { - throw new Error("Chunks cannot be empty") - } - - return await this.withClient(async client => { - const embeddingDimensions = chunks[0].embedding.length - if (embeddingDimensions === 0) { - throw new Error("Embedding dimensions must be greater than 0") - } - await this.ensureSchema(client, embeddingDimensions) - await client.query("BEGIN") - try { - const hashes = chunks.map(chunk => chunk.hash) - - await client.query( - `DELETE FROM ${this.tableName} WHERE source = $1 AND NOT (chunk_hash = ANY($2::text[]))`, - [sourceId, hashes] - ) - - const existing = await client.query( - `SELECT chunk_hash FROM ${this.tableName} WHERE source = $1 AND chunk_hash = ANY($2::text[])`, - [sourceId, hashes] - ) - const existingHashes = new Set(existing.rows.map(row => row.chunk_hash)) - - let inserted = 0 - for (const chunk of chunks) { - if (existingHashes.has(chunk.hash)) { - continue - } - await client.query( - ` - INSERT INTO ${this.tableName} (source, chunk_hash, chunk_text, embedding) - VALUES ($1, $2, $3, $4::vector) - ON CONFLICT (source, chunk_hash) DO UPDATE - SET chunk_text = EXCLUDED.chunk_text, - embedding = EXCLUDED.embedding - `, - [sourceId, chunk.hash, chunk.text, vectorLiteral(chunk.embedding)] - ) - inserted += 1 - } - - await client.query("COMMIT") - return { inserted, total: chunks.length } - } catch (error) { - await client.query("ROLLBACK") - throw error - } - }) - } - - async deleteBySourceIds(sourceIds: string[]): Promise<void> { - if (!sourceIds || sourceIds.length === 0) { - return - } - - await this.withClient(async client => { - try { - await client.query( - `DELETE FROM ${this.tableName} WHERE source = ANY($1::text[])`, - [sourceIds] - ) - } catch (error: any) { - if (error?.code === "42P01") { - // Table does not exist - return - } - throw error - } - }) - } - - async queryNearest( - embedding: number[], - sourceIds: string[], - topK: number - ): Promise<QueryResultRow[]> { - if (!sourceIds || sourceIds.length === 0) { - return [] - } - - return await this.withClient(async client => { - const { rows } = await client.query( - ` - SELECT source, chunk_text, chunk_hash, (embedding <=> $1::vector) AS distance - FROM ${this.tableName} - WHERE source = ANY($2::text[]) - ORDER BY embedding <=> $1::vector - LIMIT $3 - `, - [vectorLiteral(embedding), sourceIds, topK] - ) - - return rows.map(row => ({ - source: row.source, - chunkText: row.chunk_text, - chunkHash: row.chunk_hash, - distance: Number(row.distance ?? 1), - })) - }) - } -}
packages/server/src/sdk/workspace/ai/vectorDb/types.ts+0 −41 removed@@ -1,41 +0,0 @@ -import { VectorDbProvider } from "@budibase/types" - -interface BaseVectorDbConfig { - provider: VectorDbProvider -} - -export interface PgVectorDbConfig extends BaseVectorDbConfig { - provider: VectorDbProvider.PGVECTOR - databaseUrl: string - tableName: string -} - -export type VectorDbConfig = PgVectorDbConfig - -export interface ChunkInput { - hash: string - text: string - embedding: number[] -} - -export interface QueryResultRow { - source: string - chunkText: string - chunkHash: string - distance: number -} - -export interface VectorDb { - upsertSourceChunks( - sourceId: string, - chunks: ChunkInput[] - ): Promise<{ inserted: number; total: number }> - - deleteBySourceIds(sourceIds: string[]): Promise<void> - - queryNearest( - embedding: number[], - sourceIds: string[], - topK: number - ): Promise<QueryResultRow[]> -}
packages/server/src/sdk/workspace/ai/vectorDb/utils.ts+0 −35 removed@@ -1,35 +0,0 @@ -import { VectorDbProvider } from "@budibase/types" -import { buildPgVectorDbConfig, resolvePgVectorDbConfig } from "./pgVectorDb" -import type { VectorDb as VectorDbClient } from "./types" -import { utils } from "@budibase/shared-core" -import sdk from "../../.." - -export const createVectorDb = async ({ - namespaceId, - vectorDbId, -}: { - namespaceId: string - vectorDbId: string | undefined -}): Promise<VectorDbClient> => { - if (!vectorDbId) { - throw new Error("Vectordb id not set") - } - - const vectorDb = await sdk.ai.vectorDb.find(vectorDbId) - if (!vectorDb) { - throw new Error("Vector db not found") - } - switch (vectorDb.provider) { - case VectorDbProvider.PGVECTOR: - return buildPgVectorDbConfig(await resolvePgVectorDbConfig(vectorDb), { - namespaceId, - }) - - default: - throw utils.unreachable(vectorDb.provider, { - message: `Unsupported vector db provider: ${vectorDb.provider}`, - }) - } -} - -export * from "./types"
packages/server/src/tests/api/chatConversations.spec.ts+1 −3 modified@@ -14,7 +14,7 @@ import { quotas } from "@budibase/pro" import TestConfiguration from "../utilities/TestConfiguration" import sdk from "../../sdk" import * as agentLogs from "../../sdk/workspace/ai/agentLogs" -import type { LanguageModelV3, EmbeddingModelV3 } from "@ai-sdk/provider" +import type { LanguageModelV3 } from "@ai-sdk/provider" import { webhookChat } from "../../api/controllers/ai/chatConversations" import { MockLanguageModelV3 } from "ai/test" @@ -681,7 +681,6 @@ describe("chat conversation transient behavior", () => { sdk.ai.llm.createLLM as jest.MockedFunction<typeof sdk.ai.llm.createLLM> ).mockResolvedValue({ chat: createChatTestLanguageModel() as LanguageModelV3, - embedding: {} as EmbeddingModelV3, providerOptions: jest.fn(), uploadFile: jest.fn(), }) @@ -1060,7 +1059,6 @@ describe("Agent chat tool call tracking", () => { sdk.ai.llm.createLLM as jest.MockedFunction<typeof sdk.ai.llm.createLLM> ).mockResolvedValue({ chat: {} as any, - embedding: {} as any, providerOptions: jest.fn().mockReturnValue({}), uploadFile: jest.fn(), })
packages/types/src/api/web/global/aiConfigs.ts+0 −1 modified@@ -33,7 +33,6 @@ export interface LLMProviderField { export interface LLMProviderModels { completions: string[] - embeddings: string[] } export interface LLMProvider {
packages/types/src/api/web/global/index.ts+0 −1 modified@@ -13,5 +13,4 @@ export * from "./agents" export * from "./agentLogs" export * from "./aiConfigs" export * from "./github" -export * from "./vectorDb" export * from "./knowledgeBase"
packages/types/src/api/web/global/vectorDb.ts+0 −5 removed@@ -1,5 +0,0 @@ -import { VectorDb } from "../../../documents" - -export type VectorDbListResponse = VectorDb[] -export type CreateVectorDbRequest = Omit<VectorDb, "_id" | "_rev" | "_deleted"> -export type UpdateVectorDbRequest = VectorDb
packages/types/src/documents/document.ts+0 −1 modified@@ -50,7 +50,6 @@ export enum DocumentType { KNOWLEDGE_BASE_FILE = "knowledgebasefile", AI_CONFIG = "aiconfig", LITELLM_KEY = "litellmkey", - VECTOR_STORE = "vectordb", KNOWLEDGE_BASE = "knowledgebase", WORKSPACE_APP = "workspace_app", WORKSPACE_FAVOURITE = "workspace_favourite",
packages/types/src/documents/global/ai.ts+0 −1 modified@@ -2,7 +2,6 @@ import { Document, WebSearchConfig } from "../../" export enum AIConfigType { COMPLETIONS = "completions", - EMBEDDINGS = "embeddings", } export type ReasoningEffort = "low" | "medium" | "high"
packages/types/src/documents/global/index.ts+0 −1 modified@@ -12,5 +12,4 @@ export * from "./devInfo" export * from "./agents" export * from "./ai" export * from "./chat" -export * from "./vectorDb" export * from "./knowledgeBase"
packages/types/src/documents/global/vectorDb.ts+0 −15 removed@@ -1,15 +0,0 @@ -import { Document } from "../.." - -export enum VectorDbProvider { - PGVECTOR = "pgvector", -} - -export interface VectorDb extends Document { - name: string - provider: VectorDbProvider - host: string - port: string | number - database: string - user?: string - password?: string -}
packages/types/src/sdk/ai.ts+1 −2 modified@@ -1,4 +1,4 @@ -import type { EmbeddingModelV3, LanguageModelV3 } from "@ai-sdk/provider" +import type { LanguageModelV3 } from "@ai-sdk/provider" import { ProviderOptions } from "@ai-sdk/provider-utils" import { Readable } from "stream" @@ -105,7 +105,6 @@ export interface LLMConfigOptions { export interface LLMResponse { chat: LanguageModelV3 - embedding: EmbeddingModelV3 providerOptions?: (hasTools: boolean) => LLMProviderOptions | undefined uploadFile: ( stream: Readable,
da2003a47924Remove unused vectordb apis
12 files changed · +0 −425
packages/builder/src/stores/portal/index.ts+0 −1 modified@@ -33,7 +33,6 @@ export { bannerStore } from "./banners" export { appCreationStore } from "./appCreation" export { aiConfigsStore } from "./aiConfigs" export { translations } from "./translations" -export { vectorDbStore } from "./vectorDbs" export { aiStore } from "./ai" export const sideBarCollapsed = writable(false)
packages/builder/src/stores/portal/vectorDbs.ts+0 −48 removed@@ -1,48 +0,0 @@ -import { API } from "@/api" -import { - CreateVectorDbRequest, - UpdateVectorDbRequest, - VectorDb, -} from "@budibase/types" -import { BudiStore } from "../BudiStore" - -interface VectorDbConfigState { - configs: VectorDb[] -} - -export class VectorDbStore extends BudiStore<VectorDbConfigState> { - constructor() { - super({ - configs: [], - }) - } - - fetch = async () => { - const configs = await API.vectorDb.fetch() - this.update(state => { - state.configs = configs - return state - }) - return configs - } - - create = async (config: CreateVectorDbRequest) => { - const created = await API.vectorDb.create(config) - await this.fetch() - return created - } - - edit = async (config: UpdateVectorDbRequest) => { - const updated = await API.vectorDb.update(config) - await this.fetch() - this.update - return updated - } - - delete = async (id: string) => { - await API.vectorDb.delete(id) - await this.fetch() - } -} - -export const vectorDbStore = new VectorDbStore()
packages/frontend-core/src/api/index.ts+0 −2 modified@@ -58,7 +58,6 @@ import { buildWorkspaceFavouriteEndpoints } from "./workspaceFavourites" import { buildWorkspaceHomeEndpoints } from "./workspaceHome" import { buildRecaptchaEndpoints } from "./recaptcha" import { buildAIConfigEndpoints } from "./aiConfig" -import { buildVectorDbEndpoints } from "./vectorDbs" export type { APIClient } from "./types" @@ -332,6 +331,5 @@ export const createAPIClient = (config: APIClientConfig = {}): APIClient => { resource: buildResourceEndpoints(API), recaptcha: buildRecaptchaEndpoints(API), aiConfig: buildAIConfigEndpoints(API), - vectorDb: buildVectorDbEndpoints(API), } }
packages/frontend-core/src/api/types.ts+0 −2 modified@@ -45,7 +45,6 @@ import { WorkspaceFavouriteEndpoints } from "./workspaceFavourites" import { WorkspaceHomeEndpoints } from "./workspaceHome" import { RecaptchaEndpoints } from "./recaptcha" import { AIConfigEndpoints } from "./aiConfig" -import { VectorDbEndpoints } from "./vectorDbs" export enum HTTPMethod { POST = "POST", @@ -165,5 +164,4 @@ export type APIClient = BaseAPIClient & deployment: DeploymentEndpoints recaptcha: RecaptchaEndpoints aiConfig: AIConfigEndpoints - vectorDb: VectorDbEndpoints }
packages/frontend-core/src/api/vectorDbs.ts+0 −44 removed@@ -1,44 +0,0 @@ -import { - CreateVectorDbRequest, - UpdateVectorDbRequest, - VectorDb, - VectorDbListResponse, -} from "@budibase/types" -import { BaseAPIClient } from "./types" - -export interface VectorDbEndpoints { - fetch: () => Promise<VectorDbListResponse> - create: (config: CreateVectorDbRequest) => Promise<VectorDb> - update: (config: UpdateVectorDbRequest) => Promise<VectorDb> - delete: (id: string) => Promise<{ deleted: true }> -} - -export const buildVectorDbEndpoints = ( - API: BaseAPIClient -): VectorDbEndpoints => ({ - fetch: async () => { - return await API.get({ - url: "/api/vectordb", - }) - }, - - create: async config => { - return await API.post({ - url: "/api/vectordb", - body: config, - }) - }, - - update: async config => { - return await API.put({ - url: "/api/vectordb", - body: config, - }) - }, - - delete: async id => { - return await API.delete({ - url: `/api/vectordb/${id}`, - }) - }, -})
packages/server/src/api/controllers/ai/index.ts+0 −1 modified@@ -9,6 +9,5 @@ export * from "./configs" export * from "./chatApps" export * from "./chatConversations" export * from "./chatIdentityLinks" -export * from "./vectorDb" export * from "./files" export * from "./agentLogs"
packages/server/src/api/controllers/ai/vectorDb.ts+0 −52 removed@@ -1,52 +0,0 @@ -import { - CreateVectorDbRequest, - PASSWORD_REPLACEMENT, - UpdateVectorDbRequest, - UserCtx, - VectorDb, - VectorDbListResponse, -} from "@budibase/types" -import sdk from "../../../sdk" -import { isEnvironmentVariableKey } from "../../../sdk/utils" - -const sanitize = (config: VectorDb): VectorDb => { - if (!config.password || isEnvironmentVariableKey(config.password)) { - return config - } - return { - ...config, - password: PASSWORD_REPLACEMENT, - } -} - -export const fetchVectorDbConfigs = async ( - ctx: UserCtx<void, VectorDbListResponse> -) => { - const configs = await sdk.ai.vectorDb.fetch() - ctx.body = configs.map(sanitize) -} - -export const createVectorDbConfig = async ( - ctx: UserCtx<CreateVectorDbRequest, VectorDb> -) => { - const body = ctx.request.body - const created = await sdk.ai.vectorDb.create(body) - ctx.body = sanitize(created) - ctx.status = 201 -} - -export const updateVectorDbConfig = async ( - ctx: UserCtx<UpdateVectorDbRequest, VectorDb> -) => { - const body = ctx.request.body - const updated = await sdk.ai.vectorDb.update(body) - ctx.body = sanitize(updated) -} - -export const deleteVectorDbConfig = async ( - ctx: UserCtx<void, { deleted: true }, { id: string }> -) => { - const { id } = ctx.params - await sdk.ai.vectorDb.remove(id) - ctx.body = { deleted: true } -}
packages/server/src/api/routes/ai.ts+0 −8 modified@@ -18,10 +18,6 @@ import { createAIConfigValidator, updateAIConfigValidator, } from "./utils/validators/aiConfig" -import { - createVectorDbValidator, - updateVectorDbValidator, -} from "./utils/validators/vectorDb" import { createKnowledgeBaseValidator, updateKnowledgeBaseValidator, @@ -98,10 +94,6 @@ aiRagBuilderAdminRoutes "/api/knowledge-base/:knowledgeBaseId/files/:fileId", ai.deleteKnowledgeBaseFile ) - .get("/api/vectordb", ai.fetchVectorDbConfigs) - .post("/api/vectordb", createVectorDbValidator(), ai.createVectorDbConfig) - .put("/api/vectordb", updateVectorDbValidator(), ai.updateVectorDbConfig) - .delete("/api/vectordb/:id", ai.deleteVectorDbConfig) .get("/api/knowledge-base", ai.fetchKnowledgeBases) .post( "/api/knowledge-base",
packages/server/src/api/routes/tests/ai/vectorDb.spec.ts+0 −172 removed@@ -1,172 +0,0 @@ -import { features } from "@budibase/backend-core" -import { - FeatureFlag, - PASSWORD_REPLACEMENT, - VectorDbProvider, -} from "@budibase/types" -import TestConfiguration from "../../../../tests/utilities/TestConfiguration" -import { validatePgVectorDbConfig } from "../../../../sdk/workspace/ai/vectorDb/pgVectorDb" -import { mocks } from "@budibase/backend-core/tests" - -jest.mock("../../../../sdk/workspace/ai/vectorDb/pgVectorDb", () => { - const actual = jest.requireActual( - "../../../../sdk/workspace/ai/vectorDb/pgVectorDb" - ) - return { - ...actual, - validatePgVectorDbConfig: jest.fn().mockResolvedValue(undefined), - } -}) - -jest.mock("../../../../sdk/workspace/ai/knowledgeBase/geminiFileStore", () => { - const actual = jest.requireActual( - "../../../../sdk/workspace/ai/knowledgeBase/geminiFileStore" - ) - return { - ...actual, - createGeminiFileStore: jest.fn().mockResolvedValue("gemini_store_test"), - } -}) - -describe("vector db configs", () => { - const config = new TestConfiguration() - - afterAll(() => { - config.end() - }) - - const withRagEnabled = async <T>(f: () => Promise<T>) => { - return await features.testutils.withFeatureFlags( - config.getTenantId(), - { [FeatureFlag.AI_RAG]: true }, - f - ) - } - - const withRagDisabled = async <T>(f: () => Promise<T>) => { - return await features.testutils.withFeatureFlags( - config.getTenantId(), - { [FeatureFlag.AI_RAG]: false }, - f - ) - } - - const vectorDbRequest = { - name: "Primary Vector DB", - provider: VectorDbProvider.PGVECTOR, - host: "localhost", - port: 5432, - database: "budibase", - user: "bb_user", - password: "secret", - } - - beforeEach(async () => { - mocks.licenses.useCloudFree() - await config.newTenant() - jest.clearAllMocks() - }) - - it("creates and lists vector db configs", async () => { - await withRagEnabled(async () => { - const created = await config.api.vectorDb.create(vectorDbRequest) - expect(created._id).toBeDefined() - expect(created.password).toBe(PASSWORD_REPLACEMENT) - - const configs = await config.api.vectorDb.fetch() - expect(configs).toHaveLength(1) - expect(configs[0].name).toBe(vectorDbRequest.name) - expect(configs[0].password).toBe(PASSWORD_REPLACEMENT) - }) - }) - - it("updates and deletes vector db configs", async () => { - await withRagEnabled(async () => { - const created = await config.api.vectorDb.create(vectorDbRequest) - - const updated = await config.api.vectorDb.update({ - ...created, - name: "Updated Vector DB", - password: PASSWORD_REPLACEMENT, - }) - expect(updated.name).toBe("Updated Vector DB") - expect(updated.password).toBe(PASSWORD_REPLACEMENT) - - const { deleted } = await config.api.vectorDb.remove(created._id!) - expect(deleted).toBe(true) - - const configs = await config.api.vectorDb.fetch() - expect(configs).toHaveLength(0) - }) - }) - - it("resolves environment variable vector db settings before validating and preserves env passwords in responses", async () => { - mocks.licenses.useEnvironmentVariables() - await withRagEnabled(async () => { - await config.api.environment.create({ - name: "pg_host", - production: "prod-db.internal", - development: "dev-db.internal", - }) - await config.api.environment.create({ - name: "pg_port", - production: "6432", - development: "5433", - }) - await config.api.environment.create({ - name: "pg_password", - production: "prod-secret", - development: "dev-secret", - }) - - const created = await config.api.vectorDb.create({ - ...vectorDbRequest, - host: "{{ env.pg_host }}", - port: "{{ env.pg_port }}", - password: "{{ env.pg_password }}", - }) - - expect(validatePgVectorDbConfig).toHaveBeenCalledWith( - expect.objectContaining({ - host: "dev-db.internal", - port: 5433, - password: "dev-secret", - user: "bb_user", - database: "budibase", - }) - ) - expect(created.password).toBe("{{ env.pg_password }}") - - const configs = await config.api.vectorDb.fetch() - expect(configs[0].password).toBe("{{ env.pg_password }}") - }) - }) - - it("rejects unsupported providers", async () => { - await withRagEnabled(async () => { - await config.api.vectorDb.create( - { - ...vectorDbRequest, - provider: "pinecone" as VectorDbProvider, - }, - { status: 400 } - ) - }) - }) - - it("returns 403 when RAG is disabled", async () => { - await withRagDisabled(async () => { - await config.api.vectorDb.fetch({ status: 403 }) - await config.api.vectorDb.create(vectorDbRequest, { status: 403 }) - await config.api.vectorDb.update( - { - _id: "vectordb_test", - _rev: "1-test", - ...vectorDbRequest, - }, - { status: 403 } - ) - await config.api.vectorDb.remove("vectordb_test", { status: 403 }) - }) - }) -})
packages/server/src/api/routes/utils/validators/vectorDb.ts+0 −43 removed@@ -1,43 +0,0 @@ -import { auth } from "@budibase/backend-core" -import { VectorDbProvider } from "@budibase/types" -import Joi from "joi" - -const OPTIONAL_STRING = Joi.string().optional().allow(null).allow("") - -const PROVIDER = Joi.string() - .valid(...Object.values(VectorDbProvider)) - .required() - -const HOST = Joi.string().required() -const PORT = Joi.alternatives(Joi.number().integer(), Joi.string()).required() -const DATABASE = Joi.string().required() - -export function createVectorDbValidator() { - return auth.joiValidator.body( - Joi.object({ - name: Joi.string().required(), - provider: PROVIDER, - host: HOST, - port: PORT, - database: DATABASE, - user: OPTIONAL_STRING, - password: OPTIONAL_STRING, - }) - ) -} - -export function updateVectorDbValidator() { - return auth.joiValidator.body( - Joi.object({ - _id: Joi.string().required(), - _rev: Joi.string().required(), - name: Joi.string().required(), - provider: PROVIDER, - host: HOST, - port: PORT, - database: DATABASE, - user: OPTIONAL_STRING, - password: OPTIONAL_STRING, - }).unknown(true) - ) -}
packages/server/src/tests/utilities/api/ai/vectorDb.ts+0 −49 removed@@ -1,49 +0,0 @@ -import { - CreateVectorDbRequest, - UpdateVectorDbRequest, - VectorDb, - VectorDbListResponse, -} from "@budibase/types" -import { Expectations, TestAPI } from "../base" - -export class VectorDbAPI extends TestAPI { - fetch = async ( - expectations?: Expectations - ): Promise<VectorDbListResponse> => { - return await this._get<VectorDbListResponse>(`/api/vectordb`, { - expectations, - }) - } - - create = async ( - body: CreateVectorDbRequest, - expectations?: Expectations - ): Promise<VectorDb> => { - return await this._post<VectorDb>(`/api/vectordb`, { - body, - expectations: { - ...expectations, - status: expectations?.status || 201, - }, - }) - } - - update = async ( - body: UpdateVectorDbRequest, - expectations?: Expectations - ): Promise<VectorDb> => { - return await this._put<VectorDb>(`/api/vectordb`, { - body, - expectations, - }) - } - - remove = async ( - id: string, - expectations?: Expectations - ): Promise<{ deleted: true }> => { - return await this._delete<{ deleted: true }>(`/api/vectordb/${id}`, { - expectations, - }) - } -}
packages/server/src/tests/utilities/api/index.ts+0 −3 modified@@ -3,7 +3,6 @@ import { AIAPI } from "./ai" import { AgentAPI } from "./ai/agent" import { KnowledgeBaseAPI } from "./ai/knowledgeBase" import { KnowledgeBaseFilesAPI } from "./ai/knowledgeBaseFiles" -import { VectorDbAPI } from "./ai/vectorDb" import { AssetsAPI } from "./assets" import { AttachmentAPI } from "./attachment" import { AutomationAPI } from "./automation" @@ -68,7 +67,6 @@ export default class API { routing: RoutingAPI workspaceFavourites: WorkspaceFavouriteAPI agent: AgentAPI - vectorDb: VectorDbAPI knowledgeBase: KnowledgeBaseAPI knowledgeBaseFiles: KnowledgeBaseFilesAPI @@ -109,7 +107,6 @@ export default class API { this.routing = new RoutingAPI(config) this.workspaceFavourites = new WorkspaceFavouriteAPI(config) this.agent = new AgentAPI(config) - this.vectorDb = new VectorDbAPI(config) this.knowledgeBase = new KnowledgeBaseAPI(config) this.knowledgeBaseFiles = new KnowledgeBaseFilesAPI(config) this.public = {
cae9e1f52f1eRemove kb crud from frontend
15 files changed · +1 −1137
packages/builder/src/settings/pages/ai/knowledge-bases/files-panel/column-renderer/DeleteRenderer.svelte+0 −26 removed@@ -1,26 +0,0 @@ -<script lang="ts"> - import { AbsTooltip, ActionButton } from "@budibase/bbui" - - export let row: { - _id?: string - onDelete?: () => void - } - - const remove = () => { - row.onDelete?.() - } -</script> - -<div class="file-actions"> - <AbsTooltip text="Remove file"> - <ActionButton icon="trash" size="S" on:click={remove} /> - </AbsTooltip> -</div> - -<style> - .file-actions { - display: flex; - justify-content: flex-end; - margin-left: auto; - } -</style>
packages/builder/src/settings/pages/ai/knowledge-bases/files-panel/column-renderer/NameRenderer.svelte+0 −43 removed@@ -1,43 +0,0 @@ -<script lang="ts"> - import { KnowledgeBaseFileStatus } from "@budibase/types" - - export let row: { - filename: string - mimetype?: string - errorMessage?: string - status: KnowledgeBaseFileStatus - } -</script> - -<div class="file-name"> - <span class="file-title">{row.filename}</span> - <span class="file-meta">{row.mimetype || "text"}</span> - {#if row.status === KnowledgeBaseFileStatus.FAILED && row.errorMessage} - <span class="file-error">{row.errorMessage}</span> - {/if} -</div> - -<style> - .file-name { - display: flex; - flex-direction: column; - min-width: 0; - } - - .file-title { - font-weight: 500; - overflow: hidden; - text-overflow: ellipsis; - white-space: nowrap; - } - - .file-meta, - .file-error { - font-size: 12px; - color: var(--spectrum-global-color-gray-700); - } - - .file-error { - color: var(--spectrum-semantic-negative-color-default); - } -</style>
packages/builder/src/settings/pages/ai/knowledge-bases/files-panel/column-renderer/StatusRenderer.svelte+0 −24 removed@@ -1,24 +0,0 @@ -<script lang="ts"> - import { StatusLight } from "@budibase/bbui" - import { KnowledgeBaseFileStatus } from "@budibase/types" - - export let row: { - displayStatus: string - status: KnowledgeBaseFileStatus - } - - const getStatusProps = (status: KnowledgeBaseFileStatus) => { - switch (status) { - case KnowledgeBaseFileStatus.READY: - return { positive: true } - case KnowledgeBaseFileStatus.FAILED: - return { negative: true } - default: - return { notice: true } - } - } -</script> - -<StatusLight size="S" {...getStatusProps(row.status)}> - {row.displayStatus} -</StatusLight>
packages/builder/src/settings/pages/ai/knowledge-bases/files-panel/index.svelte+0 −256 removed@@ -1,256 +0,0 @@ -<script lang="ts"> - import { onDestroy, onMount } from "svelte" - import { - Button, - FieldLabel, - Label, - notifications, - Table, - } from "@budibase/bbui" - import { helpers } from "@budibase/shared-core" - import { - KnowledgeBaseFileStatus, - type KnowledgeBaseFile, - } from "@budibase/types" - import { knowledgeBaseStore } from "@/stores/portal" - import { createPolling } from "@/utils/polling" - import KnowledgeBaseFileDeleteRenderer from "./column-renderer/DeleteRenderer.svelte" - import KnowledgeBaseFileNameRenderer from "./column-renderer/NameRenderer.svelte" - import KnowledgeBaseFileStatusRenderer from "./column-renderer/StatusRenderer.svelte" - import { confirm } from "@/helpers" - - export interface Props { - knowledgeBaseId?: string - } - - let { knowledgeBaseId }: Props = $props() - - const FILE_STATUS_POLL_MS = 1000 - - let uploadingFile = $state(false) - let uploadError = $state("") - let fileInput = $state<HTMLInputElement>() - - let currentFiles = $derived( - ($knowledgeBaseStore.selectedKnowledgeBase?.files || []).map(f => ({ - _id: f._id, - filename: f.filename, - status: f.status, - displayStatus: formatFileStatus(f), - mimetype: f.mimetype, - errorMessage: f.errorMessage, - size: helpers.formatBytes(f.size, " "), - updatedAt: formatTimestamp(f.processedAt || f.updatedAt || f.createdAt), - onDelete: () => removeFile(f), - })) - ) - - const customRenderers = [ - { - column: "filename", - component: KnowledgeBaseFileNameRenderer, - }, - { - column: "displayStatus", - component: KnowledgeBaseFileStatusRenderer, - }, - { - column: "delete", - component: KnowledgeBaseFileDeleteRenderer, - }, - ] - - let shouldPoll = $derived( - !!knowledgeBaseId && - currentFiles.some( - file => file.status === KnowledgeBaseFileStatus.PROCESSING - ) - ) - - const poller = createPolling({ - intervalMs: FILE_STATUS_POLL_MS, - shouldPoll: () => - !!knowledgeBaseId && - currentFiles.some( - file => file.status === KnowledgeBaseFileStatus.PROCESSING - ), - poll: async () => { - if (!knowledgeBaseId) { - return - } - await knowledgeBaseStore.fetchFiles(knowledgeBaseId) - }, - }) - - const readableStatus: Record<KnowledgeBaseFileStatus, string> = { - [KnowledgeBaseFileStatus.PROCESSING]: "Processing", - [KnowledgeBaseFileStatus.READY]: "Ready", - [KnowledgeBaseFileStatus.FAILED]: "Failed", - } - - const formatFileStatus = (file: KnowledgeBaseFile) => - readableStatus[file.status] || file.status - - const formatTimestamp = (value?: string | number) => { - if (!value) { - return "—" - } - try { - return new Date(value).toLocaleString() - } catch (error) { - return value - } - } - - $effect(() => { - knowledgeBaseStore.selectKnowledgeBase(knowledgeBaseId) - }) - - $effect(() => { - if (shouldPoll) { - poller.start() - } else { - poller.stop() - } - }) - - onMount(async () => { - await knowledgeBaseStore.fetch() - }) - - async function handleFileUpload(event: Event) { - if (!knowledgeBaseId) { - return - } - const target = event.currentTarget as HTMLInputElement - const file = target?.files?.[0] - if (!file) { - return - } - - uploadError = "" - uploadingFile = true - try { - await knowledgeBaseStore.uploadFile(knowledgeBaseId, file) - notifications.success("File added to knowledge base") - } catch (error: any) { - console.error(error) - uploadError = error?.message || "Failed to upload file" - notifications.error(uploadError) - } finally { - uploadingFile = false - if (target) { - target.value = "" - } - } - } - - function handleUploadClick() { - fileInput?.click() - } - - async function removeFile(file: KnowledgeBaseFile) { - if (!knowledgeBaseId) { - return - } - await confirm({ - title: `Confirm deletion`, - body: `Are you sure to remove this file from this knowledge base? This action can't be undone.`, - okText: "Delete", - onConfirm: async () => { - try { - await knowledgeBaseStore.deleteFile(knowledgeBaseId, file._id!) - notifications.success("File removed") - } catch (error) { - console.error(error) - notifications.error("Failed to remove file") - } - }, - }) - } - - onDestroy(() => { - poller.stop() - knowledgeBaseStore.selectKnowledgeBase(undefined) - }) -</script> - -<div class="files-panel"> - <div class="files-header-row"> - <div class="files-meta"> - <FieldLabel label="Files" required /> - <Label muted - >Add text, Markdown, OpenAPI YAML, or PDF files to ground any agent - using this knowledge base.</Label - > - </div> - <div class="files-actions"> - <Button - secondary - icon="upload" - disabled={!knowledgeBaseId || uploadingFile} - on:click={handleUploadClick} - >{uploadingFile ? "Uploading..." : "Upload file"}</Button - > - <input - type="file" - accept=".txt,.md,.markdown,.json,.yaml,.yml,.csv,.tsv,.pdf" - hidden - bind:this={fileInput} - onchange={handleFileUpload} - /> - </div> - </div> - - {#if uploadError} - <div class="upload-error">{uploadError}</div> - {/if} - - {#if !knowledgeBaseId} - <Label size="L" muted> - Save the knowledge base before uploading files. - </Label> - {:else if currentFiles.length === 0} - <Label size="L" muted> - No files have been uploaded for this knowledge base yet. - </Label> - {:else} - <Table - data={currentFiles} - schema={{ - filename: { displayName: "name", width: "minmax(0, 2fr)" }, - displayStatus: { displayName: "status", width: "130px" }, - size: { width: "100px" }, - updatedAt: { displayName: "updated", width: "180px" }, - delete: { displayName: "", width: "48px", align: "Right" }, - }} - {customRenderers} - allowEditColumns={false} - allowEditRows={false} - allowSelectRows={false} - allowClickRows={false} - disableSorting - rounded - quiet - compact - ></Table> - {/if} -</div> - -<style> - .files-panel { - display: flex; - flex-direction: column; - gap: var(--spacing-s); - } - - .files-header-row { - display: flex; - align-items: flex-end; - justify-content: space-between; - } - - .upload-error { - color: var(--spectrum-semantic-negative-color-default); - } -</style>
packages/builder/src/settings/pages/ai/knowledge-bases/index.svelte+0 −201 removed@@ -1,201 +0,0 @@ -<script lang="ts"> - import InfoDisplay from "@/pages/builder/workspace/[application]/design/[workspaceAppId]/[screenId]/[componentId]/_components/Component/InfoDisplay.svelte" - import { bb } from "@/stores/bb" - import { - aiConfigsStore, - featureFlags, - knowledgeBaseStore, - vectorDbStore, - } from "@/stores/portal" - import { Button, Layout, notifications, Table } from "@budibase/bbui" - import { KnowledgeBaseType } from "@budibase/types" - import { onMount } from "svelte" - - let loading = $state(false) - - const typeToNameMap: Record<KnowledgeBaseType, string> = { - [KnowledgeBaseType.GEMINI]: "Gemini", - } - - let knowledgeBases = $derived( - [...$knowledgeBaseStore.list] - .map(kb => ({ - _id: kb._id, - name: kb.name, - type: typeToNameMap[kb.type], - files: kb.files.length, - })) - .sort((a, b) => a.name.localeCompare(b.name)) - ) - - function createKnowledgeBase() { - bb.settings(`/connections/knowledge-bases/new`) - } - - function editKnowledgeBase(row: { _id?: string }) { - if (!row._id) { - return - } - bb.settings(`/connections/knowledge-bases/${row._id}`) - } - - onMount(async () => { - loading = true - try { - await Promise.all([ - aiConfigsStore.fetch(), - knowledgeBaseStore.fetch(), - vectorDbStore.fetch(), - ]) - } catch { - notifications.error("Error fetching AI settings") - } finally { - loading = false - } - }) -</script> - -<Layout noPadding gap="XS"> - {#if $featureFlags.AI_RAG} - {#if loading} - {#each ["Knowledge bases"] as section, index} - <div class:section-spacing={index > 0}> - <div class="section-header"> - <div class="section-title">{section}</div> - <div class="skeleton-button"></div> - </div> - - <div class="skeleton-table"> - <div class="skeleton-row"> - <div class="skeleton-cell skeleton-cell--lg"></div> - <div class="skeleton-cell skeleton-cell--md"></div> - <div class="skeleton-cell skeleton-cell--sm"></div> - </div> - <div class="skeleton-row"> - <div class="skeleton-cell skeleton-cell--md"></div> - <div class="skeleton-cell skeleton-cell--lg"></div> - <div class="skeleton-cell skeleton-cell--sm"></div> - </div> - <div class="skeleton-row"> - <div class="skeleton-cell skeleton-cell--lg"></div> - <div class="skeleton-cell skeleton-cell--md"></div> - <div class="skeleton-cell skeleton-cell--sm"></div> - </div> - </div> - </div> - {/each} - {:else} - <div class="section-header"> - <div class="section-title">Knowledge bases</div> - <Button size="S" icon="plus" on:click={() => createKnowledgeBase()}> - Knowledge base - </Button> - </div> - - {#if knowledgeBases.length > 0} - <Table - compact - data={knowledgeBases} - schema={{ - name: {}, - files: { displayName: "# Files", width: "200px" }, - }} - rounded - allowClickRows={false} - allowEditRows - allowEditColumns={false} - editColumnPosition="right" - editColumnHeader="" - on:editrow={r => editKnowledgeBase(r.detail)} - ></Table> - {:else} - <InfoDisplay body="No knowledge bases created yet"></InfoDisplay> - {/if} - {/if} - {/if} -</Layout> - -<style> - .skeleton-table { - margin-top: var(--spacing-xs); - padding: 10px 14px; - border: 1px solid var(--spectrum-global-color-gray-300); - border-radius: var(--radius-l); - display: flex; - flex-direction: column; - gap: var(--spacing-s); - background: var(--spectrum-global-color-gray-25); - } - - .skeleton-row { - display: grid; - grid-template-columns: minmax(0, 1.5fr) minmax(0, 1fr) 72px; - gap: var(--spacing-m); - align-items: center; - min-height: 32px; - } - - .skeleton-cell { - height: 10px; - border-radius: 999px; - background: linear-gradient( - 90deg, - var(--spectrum-global-color-gray-200) 0%, - var(--spectrum-global-color-gray-100) 50%, - var(--spectrum-global-color-gray-200) 100% - ); - background-size: 200% 100%; - animation: skeleton-shimmer 1.2s ease-in-out infinite; - } - - .skeleton-cell--lg { - width: 100%; - } - - .skeleton-cell--md { - width: 72%; - } - - .skeleton-cell--sm { - width: 100%; - } - - .skeleton-button { - width: 132px; - height: 28px; - border-radius: 999px; - background: linear-gradient( - 90deg, - var(--spectrum-global-color-gray-200) 0%, - var(--spectrum-global-color-gray-100) 50%, - var(--spectrum-global-color-gray-200) 100% - ); - background-size: 200% 100%; - animation: skeleton-shimmer 1.2s ease-in-out infinite; - } - - @keyframes skeleton-shimmer { - 0% { - background-position: 200% 0; - } - 100% { - background-position: -200% 0; - } - } - - .section-title { - font-size: 13px; - color: var(--grey-7, #a2a2a2); - } - - .section-header { - display: flex; - justify-content: space-between; - align-items: center; - gap: var(--spacing-m); - } - - .section-spacing { - margin-top: var(--spacing-l); - } -</style>
packages/builder/src/settings/pages/ai/knowledge-bases/KnowledgeBaseForm.svelte+0 −243 removed@@ -1,243 +0,0 @@ -<script lang="ts"> - import { confirm } from "@/helpers" - import { bb } from "@/stores/bb" - import { - aiConfigsStore, - knowledgeBaseStore, - vectorDbStore, - } from "@/stores/portal" - import { Button, Helpers, Input, notifications, Select } from "@budibase/bbui" - import { - KnowledgeBaseType, - type CreateKnowledgeBaseRequest, - type UpdateKnowledgeBaseRequest, - } from "@budibase/types" - import { onMount } from "svelte" - import RouteActions from "@/settings/components/RouteActions.svelte" - import KnowledgeBaseFilesPanel from "./files-panel/index.svelte" - - export interface Props { - knowledgeBaseId: string - } - - interface KnowledgeBaseFormDraft { - _id?: string - _rev?: string - name: string - type: KnowledgeBaseType - } - - let { knowledgeBaseId }: Props = $props() - - let config = $derived( - $knowledgeBaseStore.list.find(kb => kb._id === knowledgeBaseId) - ) - - const createDraft = (): KnowledgeBaseFormDraft => - config?._id - ? { - _id: config._id, - _rev: config._rev, - name: config.name, - type: config.type, - } - : { - _id: undefined, - _rev: undefined, - name: "", - type: KnowledgeBaseType.GEMINI, - } - - let draft = $state<KnowledgeBaseFormDraft>(createDraft()) - - let isEdit = $derived(!!draft._id) - let isSaving = $state(false) - let savedSnapshot = $state<KnowledgeBaseFormDraft>() - const captureSavedSnapshot = () => { - savedSnapshot = Helpers.cloneDeep(draft) - } - captureSavedSnapshot() - let isModified = $derived( - JSON.stringify(savedSnapshot) !== JSON.stringify(draft) - ) - - let duplicateNameError = $derived.by(() => { - const normalizedDraftName = draft.name?.trim().toLowerCase() - if (!normalizedDraftName) { - return undefined - } - - const duplicate = $knowledgeBaseStore.list.find( - knowledgeBase => - knowledgeBase._id !== draft._id && - knowledgeBase.name.trim().toLowerCase() === normalizedDraftName - ) - - return duplicate - ? "A knowledge base with this name already exists" - : undefined - }) - - let canSave = $derived.by(() => { - if (isSaving || !isModified) { - return false - } - return !!draft.name?.trim() && !duplicateNameError - }) - - let knowledgeBaseTypeOptions = [ - { label: "Gemini", value: KnowledgeBaseType.GEMINI }, - ] - - onMount(async () => { - try { - await Promise.all([ - aiConfigsStore.fetch(), - knowledgeBaseStore.fetch(), - vectorDbStore.fetch(), - ]) - - const isCreation = knowledgeBaseId === "new" - if (!isCreation && !config) { - notifications.error("Knowledge base not found") - bb.settings(`/connections/knowledge-bases`) - return - } - - draft = createDraft() - captureSavedSnapshot() - } catch (err: any) { - notifications.error( - err.message || "Failed to load knowledge base settings" - ) - } - }) - - async function saveKnowledgeBase() { - draft.name = draft.name?.trim() || "" - - try { - isSaving = true - - if (draft._id && draft._rev) { - const payload: UpdateKnowledgeBaseRequest = { - _id: draft._id, - _rev: draft._rev, - name: draft.name, - type: KnowledgeBaseType.GEMINI, - } - const updated = await knowledgeBaseStore.edit(payload) - - draft._rev = updated._rev - captureSavedSnapshot() - notifications.success("Knowledge base updated") - } else { - const payload: CreateKnowledgeBaseRequest = { - name: draft.name || "", - type: KnowledgeBaseType.GEMINI, - } - - const created = await knowledgeBaseStore.create(payload) - - draft._id = created._id - draft._rev = created._rev - captureSavedSnapshot() - notifications.success("Knowledge base created") - } - bb.settings(`/connections/knowledge-bases/${draft._id}`) - } catch (err: any) { - notifications.error(err.message || "Failed to save knowledge base") - } finally { - isSaving = false - } - } - - async function deleteKnowledgeBase() { - if (!draft._id) { - return - } - - const knowledgeBaseId = draft._id - - await confirm({ - title: "Delete knowledge base", - body: "Are you sure you want to permanently delete this knowledge base?", - onConfirm: async () => { - try { - await knowledgeBaseStore.delete(knowledgeBaseId) - notifications.success("Knowledge base deleted") - bb.settings(`/connections/knowledge-bases`) - } catch (err: any) { - notifications.error(err.message || "Failed to delete knowledge base") - } - }, - }) - } -</script> - -<RouteActions> - <div class="actions"> - {#if isEdit} - <Button on:click={deleteKnowledgeBase} quiet overBackground>Delete</Button - > - {/if} - <Button on:click={saveKnowledgeBase} cta disabled={!canSave}> - {#if !isEdit} - Create knowledge base - {:else} - Save - {/if} - </Button> - </div> -</RouteActions> - -<div class="form"> - <Input - label="Display name" - description="Human readable name for the knowledge base" - required - bind:value={draft.name} - error={duplicateNameError} - placeholder="HR Policies" - /> - - {#if knowledgeBaseTypeOptions.length > 1} - <div class="select"> - <Select - label="Knowledge base type" - description="Choose where retrieval is handled." - required - bind:value={draft.type} - options={knowledgeBaseTypeOptions} - getOptionValue={option => option.value} - getOptionLabel={option => option.label} - disabled={isEdit} - /> - </div> - {/if} - - <KnowledgeBaseFilesPanel knowledgeBaseId={draft._id} /> -</div> - -<style> - .form { - display: flex; - gap: var(--spacing-s); - flex-direction: column; - } - - .actions { - display: flex; - gap: var(--spacing-s); - } - - .select { - display: flex; - gap: var(--spacing-s); - align-items: flex-end; - } - - .select :global(.spectrum-Form-item) { - flex: 1; - } -</style>
packages/builder/src/settings/pages/index.ts+0 −4 modified@@ -40,8 +40,6 @@ import Recaptcha from "@/settings/pages/recaptcha.svelte" // AI config import AIConfigsPage from "@/settings/pages/ai/completion-models/index.svelte" import AIConfigForm from "@/settings/pages/ai/completion-models/AIConfigForm.svelte" -import KnowledgeBasesPage from "@/settings/pages/ai/knowledge-bases/index.svelte" -import KnowledgeBaseForm from "@/settings/pages/ai/knowledge-bases/KnowledgeBaseForm.svelte" const componentMap = { profile: ProfilePage, @@ -58,8 +56,6 @@ const componentMap = { audit_logs: AuditLogsPage, ai_configs: AIConfigsPage, ai_config: AIConfigForm, - knowledgeBases: KnowledgeBasesPage, - knowledgeBase: KnowledgeBaseForm, auth: AuthPage, org: OrgPage, branding: BrandingPage,
packages/builder/src/settings/routes.ts+1 −25 modified@@ -1,7 +1,6 @@ import { sdk, helpers } from "@budibase/shared-core" import { AIConfigType, - FeatureFlag, GetGlobalSelfResponse, } from "@budibase/types" import { UserAvatar } from "@budibase/frontend-core" @@ -12,8 +11,7 @@ import { AdminState } from "@/stores/portal/admin" import { AppMetaState } from "@/stores/builder/app" import { PortalAppsStore } from "@/stores/portal/apps" import { StoreApp } from "@/types" -import { featureFlag } from "@/helpers" -import { aiConfigsStore, knowledgeBaseStore } from "@/stores/portal" +import { aiConfigsStore } from "@/stores/portal" import { get } from "svelte/store" const getPathId = (path: string | undefined) => { @@ -334,28 +332,6 @@ export const workspaceRoutes = ( }, ], }, - { - path: "knowledge-bases", - title: "Knowledge bases", - access: () => featureFlag.isEnabled(FeatureFlag.AI_RAG), - component: Pages.get("knowledgeBases"), - routes: [ - { - path: ":knowledgeBaseId", - component: Pages.get("knowledgeBase"), - title: (path: string | undefined) => { - const id = getPathId(path) - if (!id) { - return "New" - } - return ( - get(knowledgeBaseStore).list.find(k => k._id === id)?.name ?? - "Knowledge base" - ) - }, - }, - ], - }, ], }, {
packages/builder/src/stores/portal/index.ts+0 −1 modified@@ -34,7 +34,6 @@ export { appCreationStore } from "./appCreation" export { aiConfigsStore } from "./aiConfigs" export { translations } from "./translations" export { vectorDbStore } from "./vectorDbs" -export { knowledgeBaseStore } from "./knowledgeBases" export { aiStore } from "./ai" export const sideBarCollapsed = writable(false)
packages/builder/src/stores/portal/knowledgeBases.ts+0 −190 removed@@ -1,190 +0,0 @@ -import { API } from "@/api" -import { - CreateKnowledgeBaseRequest, - KnowledgeBase, - KnowledgeBaseFile, - UpdateKnowledgeBaseRequest, -} from "@budibase/types" -import { DerivedBudiStore } from "../BudiStore" -import { derived, Writable } from "svelte/store" - -type KnowledgeBaseWithFiles = KnowledgeBase & { - files: KnowledgeBaseFile[] -} - -interface KnowledgeBaseState { - list: KnowledgeBase[] - loading: boolean - loaded: boolean - currentKnowledgeBaseId?: string - filesByKnowledgeBaseId: Record<string, KnowledgeBaseFile[]> -} - -interface DerivedKnowledgeBaseState { - loading: boolean - loaded: boolean - currentKnowledgeBaseId?: string - list: KnowledgeBaseWithFiles[] - selectedKnowledgeBase: KnowledgeBaseWithFiles | undefined -} - -export class KnowledgeBaseStore extends DerivedBudiStore< - KnowledgeBaseState, - DerivedKnowledgeBaseState -> { - constructor() { - const makeDerivedStore = (store: Writable<KnowledgeBaseState>) => { - return derived(store, $state => { - const list = $state.list.map<KnowledgeBaseWithFiles>(knowledgeBase => ({ - ...knowledgeBase, - files: $state.filesByKnowledgeBaseId[knowledgeBase._id || ""] || [], - })) - return { - loading: $state.loading, - loaded: $state.loaded, - currentKnowledgeBaseId: $state.currentKnowledgeBaseId, - list, - selectedKnowledgeBase: $state.currentKnowledgeBaseId - ? list.find(k => k._id === $state.currentKnowledgeBaseId) - : undefined, - } - }) - } - - super( - { - list: [], - loading: false, - loaded: false, - filesByKnowledgeBaseId: {}, - }, - makeDerivedStore - ) - } - - private fetchKnowledgeBaseFiles = async (knowledgeBaseId: string) => { - const { files } = await API.knowledgeBase.fetchFiles(knowledgeBaseId) - return files - } - - private fetchFilesForKnowledgeBases = async ( - knowledgeBases: KnowledgeBase[] - ) => { - const fileEntries = await Promise.all( - knowledgeBases - .map(knowledgeBase => knowledgeBase._id) - .filter((id): id is string => !!id) - .map(async knowledgeBaseId => { - try { - return [ - knowledgeBaseId, - await this.fetchKnowledgeBaseFiles(knowledgeBaseId), - ] as const - } catch (error) { - console.error( - `Failed to fetch files for knowledge base ${knowledgeBaseId}`, - error - ) - return [knowledgeBaseId, []] as const - } - }) - ) - - return Object.fromEntries(fileEntries) - } - - fetch = async () => { - this.update(state => { - state.loading = true - return state - }) - - try { - const configs = await API.knowledgeBase.fetch() - const filesByKnowledgeBaseId = - await this.fetchFilesForKnowledgeBases(configs) - this.update(state => { - state.list = configs - state.filesByKnowledgeBaseId = filesByKnowledgeBaseId - state.loaded = true - return state - }) - return configs - } finally { - this.update(state => { - state.loading = false - return state - }) - } - } - - create = async (config: CreateKnowledgeBaseRequest) => { - const created = await API.knowledgeBase.create(config) - await this.fetch() - return created - } - - edit = async (config: UpdateKnowledgeBaseRequest) => { - const updated = await API.knowledgeBase.update(config) - await this.fetch() - return updated - } - - delete = async (id: string) => { - await API.knowledgeBase.delete(id) - this.update(state => { - delete state.filesByKnowledgeBaseId[id] - return state - }) - await this.fetch() - } - - selectKnowledgeBase = (knowledgeBaseId?: string) => { - this.update(state => { - state.currentKnowledgeBaseId = knowledgeBaseId - return state - }) - } - - fetchFiles = async (knowledgeBaseId?: string) => { - if (!knowledgeBaseId) { - return [] - } - - const files = await this.fetchKnowledgeBaseFiles(knowledgeBaseId) - this.update(state => { - state.filesByKnowledgeBaseId[knowledgeBaseId] = files - return state - }) - return files - } - - uploadFile = async (knowledgeBaseId: string, file: File) => { - const { file: uploaded } = await API.knowledgeBase.uploadFile( - knowledgeBaseId, - file - ) - this.update(state => { - state.filesByKnowledgeBaseId[knowledgeBaseId] = [ - uploaded, - ...(state.filesByKnowledgeBaseId[knowledgeBaseId] || []).filter( - existing => existing._id !== uploaded._id - ), - ] - return state - }) - return uploaded - } - - deleteFile = async (knowledgeBaseId: string, fileId: string) => { - await API.knowledgeBase.deleteFile(knowledgeBaseId, fileId) - this.update(state => { - state.filesByKnowledgeBaseId[knowledgeBaseId] = ( - state.filesByKnowledgeBaseId[knowledgeBaseId] || [] - ).filter(file => file._id !== fileId) - return state - }) - } -} - -export const knowledgeBaseStore = new KnowledgeBaseStore()
packages/frontend-core/src/api/index.ts+0 −2 modified@@ -59,7 +59,6 @@ import { buildWorkspaceHomeEndpoints } from "./workspaceHome" import { buildRecaptchaEndpoints } from "./recaptcha" import { buildAIConfigEndpoints } from "./aiConfig" import { buildVectorDbEndpoints } from "./vectorDbs" -import { buildKnowledgeBaseEndpoints } from "./knowledgeBases" export type { APIClient } from "./types" @@ -334,6 +333,5 @@ export const createAPIClient = (config: APIClientConfig = {}): APIClient => { recaptcha: buildRecaptchaEndpoints(API), aiConfig: buildAIConfigEndpoints(API), vectorDb: buildVectorDbEndpoints(API), - knowledgeBase: buildKnowledgeBaseEndpoints(API), } }
packages/frontend-core/src/api/knowledgeBases.ts+0 −79 removed@@ -1,79 +0,0 @@ -import { - CreateKnowledgeBaseRequest, - FetchKnowledgeBaseFilesResponse, - KnowledgeBase, - KnowledgeBaseFileUploadResponse, - KnowledgeBaseListResponse, - UpdateKnowledgeBaseRequest, -} from "@budibase/types" -import { BaseAPIClient } from "./types" - -export interface KnowledgeBaseEndpoints { - fetch: () => Promise<KnowledgeBaseListResponse> - create: (config: CreateKnowledgeBaseRequest) => Promise<KnowledgeBase> - update: (config: UpdateKnowledgeBaseRequest) => Promise<KnowledgeBase> - delete: (id: string) => Promise<{ deleted: true }> - fetchFiles: ( - knowledgeBaseId: string - ) => Promise<FetchKnowledgeBaseFilesResponse> - uploadFile: ( - knowledgeBaseId: string, - file: File - ) => Promise<KnowledgeBaseFileUploadResponse> - deleteFile: ( - knowledgeBaseId: string, - fileId: string - ) => Promise<{ deleted: true }> -} - -export const buildKnowledgeBaseEndpoints = ( - API: BaseAPIClient -): KnowledgeBaseEndpoints => ({ - fetch: async () => { - return await API.get({ - url: "/api/knowledge-base", - }) - }, - - create: async config => { - return await API.post({ - url: "/api/knowledge-base", - body: config, - }) - }, - - update: async config => { - return await API.put({ - url: "/api/knowledge-base", - body: config, - }) - }, - - delete: async id => { - return await API.delete({ - url: `/api/knowledge-base/${id}`, - }) - }, - - fetchFiles: async knowledgeBaseId => { - return await API.get({ - url: `/api/knowledge-base/${knowledgeBaseId}/files`, - }) - }, - - uploadFile: async (knowledgeBaseId, file) => { - const formData = new FormData() - formData.append("file", file) - return await API.post<FormData, KnowledgeBaseFileUploadResponse>({ - url: `/api/knowledge-base/${knowledgeBaseId}/files`, - body: formData, - json: false, - }) - }, - - deleteFile: async (knowledgeBaseId, fileId) => { - return await API.delete({ - url: `/api/knowledge-base/${knowledgeBaseId}/files/${fileId}`, - }) - }, -})
packages/frontend-core/src/api/types.ts+0 −2 modified@@ -46,7 +46,6 @@ import { WorkspaceHomeEndpoints } from "./workspaceHome" import { RecaptchaEndpoints } from "./recaptcha" import { AIConfigEndpoints } from "./aiConfig" import { VectorDbEndpoints } from "./vectorDbs" -import { KnowledgeBaseEndpoints } from "./knowledgeBases" export enum HTTPMethod { POST = "POST", @@ -167,5 +166,4 @@ export type APIClient = BaseAPIClient & recaptcha: RecaptchaEndpoints aiConfig: AIConfigEndpoints vectorDb: VectorDbEndpoints - knowledgeBase: KnowledgeBaseEndpoints }
packages/server/src/api/controllers/ai/index.ts+0 −1 modified@@ -10,6 +10,5 @@ export * from "./chatApps" export * from "./chatConversations" export * from "./chatIdentityLinks" export * from "./vectorDb" -export * from "./knowledgeBase" export * from "./files" export * from "./agentLogs"
packages/server/src/api/controllers/ai/knowledgeBase.ts+0 −40 removed@@ -1,40 +0,0 @@ -import { - CreateKnowledgeBaseRequest, - KnowledgeBase, - KnowledgeBaseListResponse, - UpdateKnowledgeBaseRequest, - UserCtx, -} from "@budibase/types" -import sdk from "../../../sdk" - -export const fetchKnowledgeBases = async ( - ctx: UserCtx<void, KnowledgeBaseListResponse> -) => { - const configs = await sdk.ai.knowledgeBase.fetch() - ctx.body = configs -} - -export const createKnowledgeBase = async ( - ctx: UserCtx<CreateKnowledgeBaseRequest, KnowledgeBase> -) => { - const body = ctx.request.body - const created = await sdk.ai.knowledgeBase.create(body) - ctx.body = created - ctx.status = 201 -} - -export const updateKnowledgeBase = async ( - ctx: UserCtx<UpdateKnowledgeBaseRequest, KnowledgeBase> -) => { - const body = ctx.request.body - const updated = await sdk.ai.knowledgeBase.update(body) - ctx.body = updated -} - -export const deleteKnowledgeBase = async ( - ctx: UserCtx<void, { deleted: true }, { id: string }> -) => { - const { id } = ctx.params - await sdk.ai.knowledgeBase.remove(id) - ctx.body = { deleted: true } -}
8f7e035a99afBump version to 3.35.3
1 file changed · +1 −1
lerna.json+1 −1 modified@@ -1,6 +1,6 @@ { "$schema": "node_modules/lerna/schemas/lerna-schema.json", - "version": "3.35.2", + "version": "3.35.3", "npmClient": "yarn", "concurrency": 20, "command": {
Vulnerability mechanics
Root cause
"Missing validation of the VectorDB host parameter allows SSRF to internal networks and cloud metadata endpoints."
Attack vector
An authenticated user with builder-level access sends a POST request to `/api/vectordb` (or the update endpoint) with a JSON body containing an arbitrary `host` value such as `169.254.169.254` or `localhost` [ref_id=1]. The server uses the supplied host directly to open a TCP connection, allowing the attacker to probe internal network addresses and cloud metadata endpoints that would otherwise be inaccessible [ref_id=1]. Differences in connection timing and error messages between reachable and unreachable hosts enable the attacker to enumerate internal services [ref_id=1].
Affected code
The VectorDB configuration endpoint at `/api/vectordb` accepted a `host` parameter that was validated only as a non-empty string via Joi (`Joi.string().required()`) with no allowlist or blocklist of internal IP ranges [ref_id=1]. The validator was defined in `packages/server/src/api/routes/utils/validators/vectorDb.ts` and the controller in `packages/server/src/api/controllers/ai/vectorDb.ts` forwarded the unvalidated host directly to the database SDK for connection establishment [patch_id=2725528].
What the fix does
The fix removes the entire VectorDB API surface rather than adding input validation. Patches delete the server-side controller (`vectorDb.ts`), route registrations in `ai.ts`, Joi validators (`vectorDb.ts`), the frontend API client (`vectorDbs.ts`), the builder store (`vectorDbs.ts`), and the test suite (`vectorDb.spec.ts`) [patch_id=2725528]. The commit message "Remove unused vectordb apis" indicates the feature was deemed unnecessary, eliminating the SSRF attack surface entirely [patch_id=2725528].
Preconditions
- authAttacker must have an authenticated session with builder-level access on the Budibase instance
- networkThe Budibase server must have network access to the target internal addresses (e.g., cloud metadata endpoints, internal services)
- inputAttacker sends a crafted JSON payload with an arbitrary host value to the VectorDB configuration endpoint
Reproduction
The advisory includes a Python PoC that logs in as a builder user, then iterates over target host/port pairs (e.g., `169.254.169.254:80`, `localhost:5432`, `10.0.0.1:22`) and sends POST requests to `/api/ai/vectordb` with the host and port in the JSON body, measuring response timing to infer reachability [ref_id=1].
Generated on May 27, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
1News mentions
0No linked articles in our index yet.