loadModel( ) Loads a machine learning model from a local path, remote URL, or Hyperdrive key.
This function supports multiple model types: LLM (Large Language Model), Whisper (speech recognition), embeddings, NMT (translation), and TTS. It can handle both local file paths and Hyperdrive URLs (pear://).
When `onProgress` is provided, the function uses streaming to provide real-time download progress. Otherwise, it uses a simple request-response pattern for faster execution.
function loadModel ( options : LoadModelOptions , rpcOptions ?: { forceNewConnection ?: boolean ; profiling ?: { enabled ?: boolean ; includeServerBreakdown ?: boolean ; mode ?: "summary" | "verbose" }; timeout ?: number }) : Promise Name Type Required? Description optionsLoadModelOptions✓ An object that defines all configuration parameters required for loading the model, including:
modelSrc: The location from which the model weights are fetched (local path, remote URL, or Hyperdrive URL)
modelType: The type of model ("llm", "whisper", "embeddings", "nmt", or "tts")
modelConfig: Model-specific configuration options (companion sources, model parameters, etc.)
onProgress: Callback for download progress updates
logger: Logger instance for model operation logs |
| rpcOptions | \{ forceNewConnection?: boolean; profiling?: \{ enabled?: boolean; includeServerBreakdown?: boolean; mode?: "summary" | "verbose" \}; timeout?: number \} | ✗ | Optional RPC options including per-call profiling configuration |
Error When When model loading fails, with details in the error messageWhen streaming ends unexpectedly (only when using onProgress)When receiving an invalid response type from the server
// Local file path - absolute path const localModelId = await loadModel ({ modelSrc: "/home/user/models/llama-7b.gguf" , modelType: "llm" , modelConfig: { contextSize: 2048 } }); // Local file path - relative path const relativeModelId = await loadModel ({ modelSrc: "./models/whisper-base.gguf" , modelType: "whisper" }); // Hyperdrive URL with key and path const hyperdriveId = await loadModel ({ modelSrc: "pear://<hyperdrive-key>/llama-7b.gguf" , modelType: "llm" , modelConfig: { contextSize: 2048 } }); // Remote HTTP/HTTPS URL with progress tracking const remoteId = await loadModel ({ modelSrc: "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_M.gguf" , modelType: "llm" , onProgress : ( progress ) => { console. log ( `Downloaded: ${ progress . percentage }%` ); } }); // Multimodal model with projection const multimodalId = await loadModel ({ modelSrc: "https://huggingface.co/.../main-model.gguf" , modelType: "llm" , modelConfig: { ctx_size: 512 , projectionModelSrc: "https://huggingface.co/.../projection-model.gguf" }, onProgress : ( progress ) => { console. log ( `Loading: ${ progress . percentage }%` ); } }); // Whisper with VAD model const whisperId = await loadModel ({ modelSrc: "https://huggingface.co/.../whisper-model.gguf" , modelType: "whisper" , modelConfig: { mode: "caption" , output_format: "plaintext" , min_seconds: 2 , max_seconds: 6 , vadModelSrc: "https://huggingface.co/.../vad-model.bin" } }); // Load with automatic logging - logs from the model will be forwarded to your logger import { getLogger } from "@/logging" ; const logger = getLogger ( "my-app" ); const modelId = await loadModel ({ modelSrc: "/path/to/model.gguf" , modelType: "llm" , logger // Pass logger in options });
