Meta Introduces 34B Parameter Chameleon Mixed-Modal Models

May 17, 2024, 01:20 AM

Meta has introduced Chameleon, a family of early-fusion token-based mixed-modal models capable of understanding and generating images and text in any arbitrary sequence. The research, conducted by FAIR last year, presents a stable training recipe for these fully token-based multi-modal early-fusion auto-regressive models. Chameleon, a 34B parameter autoregressive model, was trained on approximately 10 trillion tokens. The models are designed for interleaved text and image understanding and generation, showcasing state-of-the-art performance and versatile capabilities.