DrChamyoung/Quinbeta1.1

Quin Beta 1.1

Quin Beta 1.1 is an advanced Transformer-based language model with 2.7 billion parameters. It is an extension of Quin Beta 1.1, trained on an enhanced dataset that includes synthetic NLP texts and curated web content. The model demonstrates near state-of-the-art performance in benchmarks for common sense, language understanding, and logical reasoning within its parameter class. Quin Beta 1.1 is designed for QA, chat, and code formats and has a context length of 2048 tokens.

• Key Features
• Parameter Count: 2.7 billion
• Training Data: 250 billion tokens from a combination of AOAI synthetic data and filtered web data
• Training Tokens: 1.4 trillion tokens
• Hardware Used: 96xA100-80G GPUs
• Training Duration: 14 days
• Frameworks: ONNX and JAX
• Reinforcement Learning: Not applied (intended for enabling safety research)
• Applications: QA, chat, and code formats
• Context Length: 2048 tokens
• Open-Source: Yes, aimed at enabling safety research like toxicity reduction and bias understanding

Performance

Quin Beta 1.1 has demonstrated near state-of-the-art performance on various benchmarks, specifically targeting:

• Common sense reasoning
• Language understanding
• Logical reasoning

Usage

Installation

To use Quin Beta 1.1, you will need to have ONNX and JAX installed. Follow the installation instructions for each framework:

ONNX

pip install onnxruntime

JAX

pip install jax jaxlib

Loading the Model

Here is a basic example of how to load and use Quin Beta 1.1 using ONNX:

import onnxruntime as ort

# Load the ONNX model
ort_session = ort.InferenceSession("quin_beta_1.1.onnx")

# Prepare input
input_text = "Your input text here."
inputs = {
    "input_ids": [input_text],
}

# Run inference
outputs = ort_session.run(None, inputs)

# Get the result
result = outputs[0]
print(result)

Research and Safety

Quin Beta 1.1 is open-source and designed to enable research in safety domains such as:

• Toxicity reduction
• Bias understanding and mitigation

The model has not undergone reinforcement learning fine-tuning, providing a baseline for safety research efforts.

Contributing Contributions to Quin Beta 1.1 are welcome. If you would like to contribute, please follow these steps:

1. Fork the repository.
2. Create a new branch (git checkout -b feature-branch).
3. Make your changes.
4. Commit your changes (git commit -am 'Add new feature').
5. Push to the branch (git push origin feature-branch).
6. Create a new Pull Request.

License

Quin Beta 1.1 is released under the apache-2.0 License. See the LICENSE file for more details.

Contact

For any questions or issues, please contact drchamyoung@gmail.com