Skip to content

Latest commit

 

History

History
74 lines (45 loc) · 2.3 KB

ollama.md

File metadata and controls

74 lines (45 loc) · 2.3 KB

Ollama

Ollama

Ollama is an open-source project that allows users to run large language models (LLMs) on their local devices. It aims to make LLMs more accessible and affordable by providing a user-friendly interface and removing the need for expensive cloud computing resources. Ollama supports various models and offers features like model downloading, running, and fine-tuning, enabling users to customize and experiment with LLMs for a variety of applications.

Interface Name

  • ollama

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'ollama': process.env.OLLAMA_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('ollama', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: llama3
  • large: llama3
  • small: llama3

Embeddings Model Aliases

  • default: all-minilm
  • large: all-minilm
  • small: all-minilm

Options

The following parameters can be passed through options.

  • format: Details not available, please refer to the LLM provider documentation.
  • keep_alive: Details not available, please refer to the LLM provider documentation.
  • options: Details not available, please refer to the LLM provider documentation.
  • stream: If set, partial message deltas will be sent, similar to ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.

Features

  • Native JSON Mode
  • Streaming
  • Embeddings

Getting an API Key

No API Key (Local URL): This is not a traditional API so no API key is required. However, a URL(s) is required to use this service. (Ensure you have the matching models installed locally)

To get an API key, first create an Ollama account, then visit the link below.

Ollama Documentation

Ollama documentation is available here.