DeepSeek AI Unveils Janus, a 1.3B Multimodal Model With Decoupled Visual Encoding and Image Generation

Oct 18, 2024, 08:08 AM

DeepSeek AI, in collaboration with researchers from the University of Hong Kong and Peking University, has unveiled Janus, a 1.3 billion parameter multimodal model that integrates image generation capabilities. Janus is designed as an autoregressive framework that unifies multimodal understanding and generation by decoupling visual encoding, using different visual encoders for understanding and generation, which enhances flexibility and performance. The model is built upon DeepSeek-LLM-1.3b-base and incorporates SigLIP-L as its vision encoder. Despite its advanced capabilities, Janus is super small in size, only 1.8 billion parameters. Utilizing a single transformer architecture, Janus is trained on approximately 500 billion text tags and employs a specific tokenizer for image generation with a downsampling rate of 16. As DeepSeek AI's first multimodal offering on Hugging Face, the model is now available for download.