v5 Eagle 7B HF EU AI Act Compliance Profile

Huggingface RWKV-5 Eagle 7B Model - via HF Transformers Library

! Important Note !

The following is the HF transformers implementation of the RWKV-5 Eagle 7B model. This is meant to be used with the huggingface transformers

For the full model weights on its own, to use with other RWKV libraries, refer to here

This is not an instruct tune model! (soon...)

• HF Demo
• Our wiki
• pth model weights

Running on CPU via HF transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def generate_prompt(instruction, input=""):
    instruction = instruction.strip().replace('\r\n','\n').replace('\n\n','\n')
    input = input.strip().replace('\r\n','\n').replace('\n\n','\n')
    if input:
        return f"""Instruction: {instruction}

Input: {input}

Response:"""
    else:
        return f"""User: hi

Assistant: Hi. I am your assistant and I will provide expert full response in full details. Please feel free to ask any question and I will always answer it.

User: {instruction}

Assistant:"""


model = AutoModelForCausalLM.from_pretrained("RWKV/HF_v5-Eagle-7B", trust_remote_code=True).to(torch.float32)
tokenizer = AutoTokenizer.from_pretrained("RWKV/HF_v5-Eagle-7B", trust_remote_code=True)

text = "请介绍北京的旅游景点"
prompt = generate_prompt(text)

inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(inputs["input_ids"], max_new_tokens=333, do_sample=True, temperature=1.0, top_p=0.3, top_k=0, )
print(tokenizer.decode(output[0].tolist(), skip_special_tokens=True))

output:

Your obligations as a v5 Eagle 7B HF deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library