Ollama (executable & models) install location

[jandubois]

There are 2 scenarios: (1) the user already has Ollama installed, or (2) they do not.

In the first case I think we would always re-use the existing installation. That means additional downloaded models will be merged in with their other models, and we will never uninstall Ollama nor remove the models.

If the user doesn't have an Ollama install, then there are 2 options again: (1) We install Ollama and the models in the default location, just like above, or (2) we install them into a Rancher Desktop specific location that will be deleted when the user uninstalls the extension (or performs a factory reset).

I do think that we need to offer both choices here.

I think the clearest way to implement this would be to postpone the Ollama installation until the user opens the extension for the first time. If Ollama is not yet installed, the user will get a screen giving them both options, and explaining the consequences of each option.

This is somewhat similar to how the Epinio extension makes you install the server part explicitly:

[mook-as]

In case 1), how do we figure out where the Ollama executable is? The user is more likely to run the installer on Windows, but on mac it's probably a .app (and Linux is just an archive you unpack somewhere). There is no "default location".

Reusing the data (models etc.) is easier.

I think that means we can delay running the installer (to what we already have before the hooks), but still run the uninstaller. The uninstaller wouldn't have any UI, but that should be fine.

[jandubois]

on mac it's probably a .app

It is indeed an .app, but it seems to be just a giant Electron app to install the CLI. It will show you a sudo prompt to create a symlink:

$ ls -l $(which ollama)
lrwxr-xr-x  1 root  staff  50 25 Sep 19:05 /usr/local/bin/ollama -> /Applications/Ollama.app/Contents/Resources/ollama

So we could check the following "known" locations:

/usr/local/bin/ollama
/Applications/Ollama.app/Contents/Resources/ollama
$HOME/Applications/Ollama.app/Contents/Resources/ollama

If we find any of those, we can know it is already installed.

If you want to go the extra mile, you could check if it is currently running:

$ lsof -i :11434
COMMAND  PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
ollama  2856  jan    3u  IPv4 0x2543887a5528b965      0t0  TCP localhost:11434 (LISTEN)

$ ps -p 2856 -o comm=
/Applications/Ollama.app/Contents/Resources/ollama

On Linux the install script seems to try to install into /usr/local/bin, /usr/bin, or /bin.

[mook-as]

We already check if an existing instance is running:
https://github.com/rancher-sandbox/rd-open-webui-docker-ext/blob/604933cc3c6159ba3631dba8554da25eb5aedd09/installer/main.go#L18
https://github.com/rancher-sandbox/rd-open-webui-docker-ext/blob/604933cc3c6159ba3631dba8554da25eb5aedd09/installer/main.go#L31-L32

[gunamata]

While I agree that we should implement this in a way to not interfere/overwrite an existing Ollama installation, I am not sure about the benefits of offering a choice to install in a normal location or a Rancher Desktop chosen location, keeping in mind the additional efforts required to develop/maintain the two options.

IMO, below behavior should work for most people.

if Ollama is already available on the host machine, just use it and don't do anything to Ollama on the extension uninstall
if Ollama is not available then install it in a RD specific location and clean that up on the extension uninstall

[jandubois]

if Ollama is already available on the host machine, just use it and don't do anything to Ollama on the extension uninstall

That's what I wrote at the beginning and I think it is non-controversial. The only questions is how hard should we look for it if it is not currently running.

if Ollama is not available then install it in a RD specific location and clean that up on the extension uninstall

I agree with installation into an RD specific location, if for no other reason than that the default location may require a sudo prompt. 😄

I think the big question is about cleaning up the models. The tinyllama model takes over 600MB of disk space, which is a bit much to just leave behind in a hidden directory.

Maybe we just remove the models if tinyllama is the only model that is installed, but leave them behind if the user installed additional ones. It would be nice if we could tell the user about it somehow.

---

Another random concern I had: installing models into ~/.ollama means that they will be automatically be added to Time Machine backups. 600MB is maybe not a concern, but larger models might be. On the other hand, this is an Ollama issue and not a Rancher Desktop one. But again it would be nice to be able to tell the user about it.

So maybe having an "intro" page for the extension would still be nice, even if we don't offer any choices. You get to see the page every time you launch the extension, and you have to click an "OpenWebUI" button to actually display the app?