Finding config.json for Llama 3.1 8B

Question

I installed the Llama 3.1 8B model through Meta's Github page, but I can't get their example code to work. I'm running the following code in the same directory as the Meta-Llama-3.1-8B folder:

import transformers
import torch

pipeline = transformers.pipeline(
  "text-generation",
  model="Meta-Llama-3.1-8B",
  model_kwargs={"torch_dtype": torch.bfloat16},
  device="cuda"
)

The error is

OSError: Meta-Llama-3.1-8B does not appear to have a file named config.json

Where can I get config.json?

I've installed the latest transformers module, and I understand that I can access the remote model on HuggingFace. But I'd rather use my local model. Is this possible?

desertnaut · Accepted Answer · 2024-08-23 19:26:12Z

5

The issue isn't on your end. The confusion arises from Meta not clearly distinguishing between the distributions via Hugging Face and download.sh.

To resolve this, you can download the model files using the Hugging Face CLI:

!huggingface-cli download meta-llama/Meta-Llama-3-8B-Instruct --local-dir meta-llama/Meta-Llama-3-8B-Instruct

This method will provide you with the config.json and tokenizer.json files.

Additionally, you can try downloading other versions manually. For instance, someone shared a link to the configuration file on Hugging Face:

llama-3-8b/config.json

edited Aug 23, 2024 at 19:26

desertnaut

60.8k32 gold badges155 silver badges183 bronze badges

answered Aug 4, 2024 at 22:13

Z'K

914 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Finding config.json for Llama 3.1 8B

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related