Post
3436
Pebble-25M and Pebble-25M-Chat are out now!
We’re excited to release Pebble-25M and Pebble-25M-Chat!
Both models use our 3:1 Mamba2/Transformer hybrid architecture and were pretrained on 25B tokens. Pebble-25M-Chat was then further fine-tuned on an additional 250M tokens from smol-smoltalk, following the same approach used for the Pebble-10M models.
We hope you enjoy experimenting with them!
Pebble-50M is coming in a few days.
Models
Pebble-25M: basically-ai/Pebble-25M
Pebble-25M-Chat: basically-ai/Pebble-25M-Chat
Pebble-10M GGUFs
In case you missed it, our friend @ContextReq made GGUF versions of the Pebble-10M models:
https://huggingface.co/ContextReq/Pebble-10M-GGUF
https://huggingface.co/ContextReq/Pebble-10M-Chat-GGUF
Follow us if you don’t want to miss future releases and updates!
@Hoglet-33
basically-ai
basically-experimental
We’re excited to release Pebble-25M and Pebble-25M-Chat!
Both models use our 3:1 Mamba2/Transformer hybrid architecture and were pretrained on 25B tokens. Pebble-25M-Chat was then further fine-tuned on an additional 250M tokens from smol-smoltalk, following the same approach used for the Pebble-10M models.
We hope you enjoy experimenting with them!
Pebble-50M is coming in a few days.
Models
Pebble-25M: basically-ai/Pebble-25M
Pebble-25M-Chat: basically-ai/Pebble-25M-Chat
Pebble-10M GGUFs
In case you missed it, our friend @ContextReq made GGUF versions of the Pebble-10M models:
https://huggingface.co/ContextReq/Pebble-10M-GGUF
https://huggingface.co/ContextReq/Pebble-10M-Chat-GGUF
Follow us if you don’t want to miss future releases and updates!
@Hoglet-33
