Hugging Face's logo

alpha-ai
/
AlphaAI-Chatty-INT1

Transformers
GGUF
English
llama
text-generation-inference
unsloth
conversational
Model card Files
xet
 Community
1
How to use from
llama.cpp
Install from brew 
brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf alpha-ai/AlphaAI-Chatty-INT1:
# Run inference directly in the terminal:
llama-cli -hf alpha-ai/AlphaAI-Chatty-INT1:
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf alpha-ai/AlphaAI-Chatty-INT1:
# Run inference directly in the terminal:
llama-cli -hf alpha-ai/AlphaAI-Chatty-INT1:
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf alpha-ai/AlphaAI-Chatty-INT1:
# Run inference directly in the terminal:
./llama-cli -hf alpha-ai/AlphaAI-Chatty-INT1:
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf alpha-ai/AlphaAI-Chatty-INT1:
# Run inference directly in the terminal:
./build/bin/llama-cli -hf alpha-ai/AlphaAI-Chatty-INT1:
Use Docker 
docker model run hf.co/alpha-ai/AlphaAI-Chatty-INT1:
Quick Links
Title card

Website - https://www.alphaai.biz

Uploaded model

  • Developed by: alphaaico
  • License: apache-2.0
  • Finetuned from model : llama-3.2-3b-instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

AlphaAI-Chatty-INT1

Overview AlphaAI-Chatty-INT1 is a fine-tuned LLaMA 3B Small model optimized for chatty and engaging conversations. This model has been trained on a proprietary conversational dataset, making it well-suited for local deployments that require a natural, interactive dialogue experience.

The model is available in GGUF format and has been quantized to different levels to support various hardware configurations.

Model Details

  • Base Model: LLaMA 3B Small
  • Fine-tuned By: Alpha AI
  • Training Framework: Unsloth

Quantization Levels Available:

Format: GGUF (Optimized for local deployments)

Use Cases:

  • Conversational AI – Ideal for chatbots, virtual assistants, and customer support.
  • Local AI Deployments – Runs efficiently on local machines without requiring cloud-based inference.
  • Research & Experimentation – Suitable for studying conversational AI and fine-tuning on domain-specific datasets.

Model Performance The model has been optimized for chat-style interactions, ensuring:

  • Engaging and context-aware responses
  • Efficient performance on consumer hardware
  • Balanced coherence and creativity in conversations

Limitations & Biases This model, like any AI system, may have biases from the training data. It is recommended to use it responsibly and fine-tune further if needed for specific applications.

License This model is released under a permissible license. Please check the Hugging Face repository for more details.

Acknowledgments Special thanks to the Unsloth team for providing an optimized training pipeline for LLaMA models.

Downloads last month
43
GGUF
Model size
3B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including alpha-ai/AlphaAI-Chatty-INT1