AzeriLLM (unt-7b)
| Property | Value |
|---|---|
| Model Size | 7 Billion parameters |
| Base Architecture | LLaMA |
| Author | Omar Ibrahim |
| Model URL | HuggingFace |
What is unt-7b?
AzeriLLM (unt-7b) is a groundbreaking language model specifically designed for the Azerbaijani language, built upon the LLaMA architecture. Developed by 17-year-old NLP researcher Omar Ibrahim, it represents the first dedicated large language model for Azerbaijani, achieving remarkable performance metrics through an innovative training approach.
Implementation Details
The model employs a sophisticated training pipeline featuring synthetic data creation with 2.7M bilingual (EN↔AZ) segments, custom NLLB translation, and Direct Preference Optimization (DPO). This comprehensive approach has resulted in significant improvements over the base LLaMA model, with perplexity reduced from 42.3 to 13.4 and BLEU scores for English-to-Azerbaijani translation reaching 36.7.
- Custom NLLB Translation optimization for Azerbaijani
- DPO-enhanced training for human-level quality
- 2.7M bilingual training segments
- Improved fluency and coherence metrics
Core Capabilities
- High-quality Azerbaijani language generation
- Advanced English-Azerbaijani translation capabilities
- Superior fluency (4.7/5.0 in human evaluation)
- Enhanced factual accuracy (4.5/5.0 in human evaluation)
- Coherent text generation (4.6/5.0 in human evaluation)
Frequently Asked Questions
Q: What makes this model unique?
This is the first Azerbaijani-focused LLM based on the LLaMA architecture, specifically optimized for Azerbaijani language processing through a novel training pipeline combining synthetic data and DPO.
Q: What are the recommended use cases?
The model is ideal for Azerbaijani text generation, translation tasks, and general language understanding applications. It's particularly suited for scenarios requiring high fluency and accuracy in Azerbaijani language processing.
