llama3 llava next 8b hf EU AI Act Compliance Profile

LLaVa-Next Model Card

The LLaVA-NeXT model was proposed in LLaVA-NeXT: Stronger LLMs Supercharge Multimodal Capabilities in the Wild by Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, Chunyuan Li. These LLaVa-NeXT series improves upon LLaVa-1.6 by training with stringer language backbones, improving the performance.

Disclaimer: The team releasing LLaVa-NeXT did not write a model card for this model so this model card has been written by the Hugging Face team.

Model description

LLaVa combines a pre-trained large language model with a pre-trained vision encoder for multimodal chatbot use cases. LLaVA NeXT Llama3 improves on LLaVA 1.6 BY:

• More diverse and high quality data mixture
• Better and bigger language backbone

Base LLM: meta-llama/Meta-Llama-3-8B-Instruct

Intended uses & limitations

You can use the raw model for tasks like image captioning, visual question answering, multimodal chatbot use cases. See the model hub to look for other versions on a task that interests you.

How to use

To run the model with the pipeline, see the below example:

from transformers import pipeline

pipe = pipeline("image-text-to-text", model="llava-hf/llama3-llava-next-8b-hf")
messages = [
    {
      "role": "user",
      "content": [
          {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"},
          {"type": "text", "text": "What does the label 15 represent? (1) lava (2) core (3) tunnel (4) ash cloud"},
        ],
    },
]

out = pipe(text=messages, max_new_

Your obligations as a llama3 llava next 8b hf deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library