AIhomeJP/home1.1-chat

AIhomeJP Home1.1 Chat Inference

A minimal example for running inference with the AIhomeJP/home1.1-chat model using the Hugging Face Transformers library.

Requirements

• Python 3.10+
• PyTorch
• Transformers

Install the dependencies:

pip install torch transformers

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AIhomeJP/home1.1-chat"

device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
)

# Compatibility workaround
model.config.num_hidden_layers = 1

model.to(device)
model.eval()

system_prompt = (
    "You are a helpful, honest, and harmless AI assistant. "
    "Answer clearly and concisely."
)

user_prompt = "Hello!"

prompt = (
    f"<|system|>{system_prompt}<|end|>"
    f"<|user|>{user_prompt}<|end|>"
    f"<|assistant|>"
)

inputs = tokenizer(prompt, return_tensors="pt").to(device)

# Required workaround
inputs.pop("attention_mask", None)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        use_cache=False,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Features

• Supports system and user prompts.
• Automatically selects CUDA when available.
• Generates only the assistant response.
• Compatible with AIhomeJP/home1.1-chat.

Notes

This example applies the following compatibility workarounds:

• Sets model.config.num_hidden_layers = 1.
• Removes the attention_mask before generation.
• Uses use_cache=False during generation.

These settings are intended for compatibility with the current model implementation.

Output

Example:

User:
Hello!

Assistant:
Hello! How can I help you today?

License

Please refer to the model repository for the applicable model license and usage terms.