OllamaAPITestProject

A basic DevContainer setup and web page to test API responses using Ollama.

Non-gpu Setup

For use without a GPU (E.g. GitHub codespaces, and macOS) please:

• Rename .devcontainer/docker-compose_CPU.yaml to .devcontainer/docker-compose.yaml.
• In TestAPI.sh change localhost:11434 to ollama11434

GitHub Codespaces

The containers currently build and run, but there are CORS errors when trying to interact with Ollama's API.

Required Tools

This repo is setup for use with NVIDIA cards and Docker containers.

1. Install the NVIDIA Container Toolkit (CTK) on the host device.

WSL2 Users take note you may need to run an extra command to configure the CTK.
sudo nvidia-ctk runtime configure --runtime=docker --config /mnt/<windows drive>/Users/<your user>/.docker/daemon.json

2. Install Docker/Docker Desktop on the host device.

Running the web server

1. cd ollamaapp
2. npm install
3. node index.js

Navigate to localhost:3000 on the host machine. Ollama is running on port 11434 in a separate container.

Using the sample web page.

1. Pull the model specified in the MODEL variable (index.html).
2. Enter a prompt into the text box.
3. Click Submit and wait for the output to be displayed on the page.

A RTX 2060, Ryzen 5 2600, 24GB DDR4 took ~4 minutes to create a response when first run. Subsequent runs do not take as long.

Additional Resources

• Ollama API Reference
• ollama/ollama
• valiantlynx/ollama-docker