A way to keep the model longer around?

[gerroon]

Is there a setting to make sure Ollama keeps the model in memory longer? A way to set a time range?

[delfireinoso]

If you run only one model only you can tell Ollama to keep it loaded in Memory for a longer time after the model request with the parameter Keep_alive (default 5 minutes)

There is no indication but you can try time in seconds and pass the parameter when you first run Ollama in your system

[gerroon]

How do I do that with Perplexica? I use Ollama with other apps, so I need this only for Perplexixa.

[delfireinoso]

You can try to set it on Ollama at run time

The models are pulled and the default time is applicable for this session

If you set a longer time it will set for all apps but any other model pull will reset the time

For example you can use Ollama for a lot of apps

But you run Ollama first so

Ollama keep_alive=600

should wait 10 minutes to unload the model and free memory

Every time you use Ollama with the API the time is reset to 10 minutes

[ItzCrazyKns]

There is a setting in the config.toml file called KEEP_ALIVE, you can use it to specify how long you want the model to be loaded for. https://github.com/ItzCrazyKns/Perplexica/blob/master/sample.config.toml#L4