Loading...
Loading...
Browse all stories on DeepNewz
VisitMistralAI Releases New 25.38 GB Pixtral-12B Vision-Language Model
Sep 11, 2024, 07:01 AM
MistralAI has released a new vision-language multimodal model called Pixtral-12B. The model, which is available via a magnet link, has a size of 25.38 GB. Key architectural features of Pixtral-12B include a dimension of 5120, 40 layers, a head dimension of 128, a hidden dimension of 14336, 32 heads, 8 key-value heads, a rope theta of 1000000000.0, a normalization epsilon of 1e-05, and a vocabulary size of 131072. The model also incorporates GeLU and 2D RoPE for the vision adapter and includes three new tokens in its tokenizer. The vision encoder's hidden size is also notable.
View original story
Markets
Yes • 50%
No • 50%
Official announcements from major tech companies or MistralAI
No • 50%
Yes • 50%
Publicly available benchmark results or research papers
No • 50%
Yes • 50%
Official announcements from consumer application companies or MistralAI
21-30 • 25%
31+ • 25%
0-10 • 25%
11-20 • 25%
Publicly available research databases such as arXiv or Google Scholar
Top 6-10 • 25%
Top 1 • 25%
Outside Top 10 • 25%
Top 2-5 • 25%
Publicly available benchmark results from reputable sources
Other • 25%
Healthcare • 25%
Automotive • 25%
Retail • 25%
Official announcements from MistralAI or major tech companies