Aurora1.0-150M has just been released as the first generation of a... - Vibeus
Aurora1.0-150M has just been released as the first generation of a 150M model. Its performance is described as similar to GPT2-Small.
Reported benchmark scores:
PIQA: 62.24%
Hellaswag: 32.20%
Arc-Easy: 44.91%
Arc-Challenge: 25.00%
Arithmark 3.0: 33.90%
CapitalBench: 36.55%
The model was trained on 7B tokens using an RTX Pro 6000. An example inference script is available in the Hugging Face repository.