Skip to content
AI in a Box

Troubleshooting Guide

If you’re reading this, let me know what happened so I can make this better. That being said, here’s a few things that might help you troubleshoot.

Sometimes it’s hard to know where to start. If you’re unsure, start by tailing the logs.

Terminal window
docker compose logs --follow

Docker

This whole system is built on the docker compose.

To verify that docker is installed and running, run the following command:

Verify Docker is Installed

Terminal window
docker --version

Verify Docker Containers Running

Terminal window
docker ps

You should see something like:

Terminal window
CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                                       NAMES
5405f771f6bf   test-app                        "docker-entrypoint.s…"   12 minutes ago   Up 12 minutes   8080/tcp                                    app
ebe88b2812eb   chromadb/chroma:latest          "/docker_entrypoint.…"   12 minutes ago   Up 12 minutes   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp   chromadb
6442e460f54f   ollama/ollama:latest            "/bin/ollama serve"      12 minutes ago   Up 12 minutes   11434/tcp                                   ollama

Start Docker Containers

Depending on your system running ./start.sh or ./start.bat should start the containers.

If that fails, you can try to start the containers manually.

Using GPU

Terminal window
docker compose -f dockercompose-app.yaml -f dockercompose-chroma.yaml -f dockercompose-ollama-gpu.yaml up -d

Using CPU

Terminal window
docker compose -f dockercompose-app.yaml -f dockercompose-chroma.yaml -f dockercompose-ollama-cpu.yaml up -d

Ollama

You can test the ollama container by popping on to the container and running a model

Testing a model

Terminal window
docker exec -it ollama /bin/bash
ollama run tinydolphin

Express App

This is the interface to the rest of the system.

Testing the app

If you aren’t getting a response from the express app, you can test it by running the following command:

(curl isn’t installed in the container, so you’ll have to use the node repl)

Terminal window
docker exec -it app /bin/bash
node
await fetch('http://app:8080/summarizeTask', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    "text": "What is the capital of France?"
    //"text": text
  })
}).then(res => res.json()).then(data => console.log(data));

That should return something like;

{
  model: 'tinydolphin',
  created_at: '2024-02-28T16:47:32.765024841Z',
  response: ' The capital of France is Paris.',
  done: true,
  context: [ ... ],
  total_duration: 302190711,
  load_duration: 303892,
  prompt_eval_duration: 39757000,
  eval_count: 8,
  eval_duration: 261758000
}