all-MiniLM-L6-v2
| Property | Value |
|---|---|
| Parameter Count | 22.7M |
| Output Dimensions | 384 |
| License | Apache 2.0 |
| Framework Support | PyTorch, TensorFlow, ONNX, Rust |
What is all-MiniLM-L6-v2?
all-MiniLM-L6-v2 is a powerful sentence embedding model that transforms text into 384-dimensional vector representations. Developed by sentence-transformers, it's specifically designed for semantic similarity tasks and has been trained on an impressive dataset of over 1 billion sentence pairs.
Implementation Details
The model is built upon the pretrained nreimers/MiniLM-L6-H384-uncased architecture and fine-tuned using a contrastive learning objective. Training was conducted on TPU v3-8 hardware with a batch size of 1024 over 100k steps, using the AdamW optimizer with a 2e-5 learning rate.
- Efficient architecture with only 22.7M parameters
- 384-dimensional dense vector output
- Maximum sequence length of 256 tokens
- Trained on 21 diverse datasets including Reddit comments, WikiAnswers, and academic papers
Core Capabilities
- Sentence and paragraph embedding generation
- Semantic similarity computation
- Text clustering
- Information retrieval tasks
- Cross-lingual text matching
Frequently Asked Questions
Q: What makes this model unique?
The model's uniqueness lies in its efficient architecture combined with extensive training on over 1 billion sentence pairs from diverse sources, making it particularly robust for real-world applications while maintaining a small parameter footprint.
Q: What are the recommended use cases?
The model excels in tasks requiring semantic understanding such as document similarity comparison, semantic search, clustering of text documents, and information retrieval. It's particularly well-suited for applications where computational efficiency is important.
