Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local LLM: Use OSProcessHandler.Silent intead of OSProcessHandler #287

Merged

Conversation

ViktorGozhiy
Copy link
Contributor

If after several minutes of inactivity you send a message to the chat, the plugin crashes. The logs contain the following:

2023-11-22 16:12:33,346 [ 317844]   WARN - #c.i.e.p.BaseOSProcessHandler - Process hasn't generated any output for a long time.
If it's a long-running mostly idle daemon process, consider overriding OSProcessHandler#readerOptions with 'BaseOutputReader.Options.forMostlySilentProcess()' to reduce CPU usage.
Command line: ./server -m /home/user/.codegpt/models/gguf/codellama-7b-instruct.Q4_K_M.gguf -c 2048 --port 39193 -t 20
java.lang.Throwable: Process creation:
	at com.intellij.execution.process.BaseOSProcessHandler.<init>(BaseOSProcessHandler.java:33)
	at com.intellij.execution.process.OSProcessHandler.<init>(OSProcessHandler.java:45)
	at ee.carlrobert.codegpt.completions.llama.LlamaServerAgent$1.processTerminated(LlamaServerAgent.java:90)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.intellij.execution.process.ProcessHandler$2.invoke(ProcessHandler.java:248)
	at jdk.proxy2/jdk.proxy2.$Proxy53.processTerminated(Unknown Source)

The IDE kills the server process.
To prevent this from happening, you must use the OSProcessHandler.Silent class to start the llama.cpp server.
This pull request contains the relevant changes.

@carlrobertoh carlrobertoh merged commit 1acb950 into carlrobertoh:master Nov 22, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants