0

Hi I'm using Jupyter Notebook, and trying to create instance of llama-2-7b-chat.q4_K_M.gguf (this is a quantized model) from hugging face. I'm running the following code:

from langchain_community.llms import CTransformers

llm = CTransformers(model="./models/llama-2-7b-chat.q4_K_M.gguf")

And when I run it my interface immediately says The kernel for Documents/llm_rcf/langchain_ctransformer_trial.ipynb appears to have died. It will restart automatically.

The terminal says the following :

[I 2024-04-16 11:58:17.119 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports
[W 2024-04-16 11:58:17.120 ServerApp] kernel 7483deb0-17e0-4a4e-bce4-cc5f4d26fef9 restarted
[I 2024-04-16 11:58:17.134 ServerApp] Starting buffering for 7483deb0-17e0-4a4e-bce4-cc5f4d26fef9:18a4e018-29ef-47eb-8564-e33410b825af
[I 2024-04-16 11:58:17.195 ServerApp] Connecting to kernel 7483deb0-17e0-4a4e-bce4-cc5f4d26fef9.
[I 2024-04-16 11:58:17.196 ServerApp] Restoring connection for 7483deb0-17e0-4a4e-bce4-cc5f4d26fef9:18a4e018-29ef-47eb-8564-e33410b825af
[I 2024-04-16 11:58:26.144 ServerApp] Saving file at /Documents/llm_rcf/langchain_ctransformer_trial.ipynb

I have no idea whats going, would appreciate if anyone can help I'm using Apple's Mac Book Pro M1 Chip - 32 GB RAM and when I'm running it my activity monitor says Memory Used: 12.05GB, Physical Memory: 32.00GB

Improve this question
2
  • I encountered a similar issue and managed to resolve it by utilizing the GPT4ALL and OLLAMA libraries from GitHub. Additionally, I found success by switching from Jupyter to a Python file in Visual Studio Code. Commented Apr 16, 2024 at 3:36
  • @MubashirAhmedSiddiqui I did switch to VS code but I'm getting 'Illegal Instruction 4" and there is no traceback, so I can't figure whats the issue Commented Apr 16, 2024 at 5:57
Related questions
24
How to debug dying Jupyter Python3 kernel?
1
Hugging Face Model import error to Jupyter Notebook :
1
Ipython/Jupyter: Timeout waiting for kernel_info reply
Load 2 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.