The Stable Diffusion 3 API is available now, and work on Stable Assistant is getting closer

After just a couple months of previews, Stability AI is introducing the next generation of its generative artificial intelligence model, Stable Diffusion 3. Along with the update, Stability AI is also introducing a preview of a new chatbot technology called "Stable Assistant."

Stable Diffusion 3 was first announced as a preview version back in February. As of today, Stable Diffusion 3 is available for use through an API on the Stability AI developer platform. As an API, Stable Diffusion can be integrated into services and applications that utilize the text-to-image generation capabilities that the model provides. In addition to the base model, the Stable Diffusion 3 Turbo model is now available.

In Stable Diffusion 3, Stability AI has implemented a number of new machine learning and artificial intelligence technologies to improve image generation as well as typography. The company's key goal in releasing the API was to make it production-ready.

"We have implemented a number of protective measures to help prevent misuse of SD3, and we continue to refine these measures based on user feedback," Christian Laforte, CTO and interim CEO of Stability AI, told VentureBeat in an exclusive interview.

Join enterprise leaders in San Francisco July 9-11 at our flagship AI event. Network with peers, explore the opportunities and challenges of generative AI, and learn how to integrate AI applications into your industry.

An open model is on the way, but it's not soon enough

While Stable Diffusion 3 is already available via API, there is no open model that is publicly available yet, but there will be.

"We will continually work to improve the model before its open release," said LaForte. "In line with our commitment to open generative AI, we intend to soon make the scale model available for self-hosting with Stability AI membership."

Stability AI membership is a strategy the company first announced in December to help it build a new revenue model.

Fireworks will help power and operate the Stable Diffusion 3 APIs

The Stable Diffusion 3 API deserves special attention as it will benefit from Stability AI's partnership with API platform provider Fireworks AI.

Ensuring full API performance for artificial intelligence applications can be a challenge, especially when it comes to scale. This is exactly the challenge that Fireworks AI can help you solve.

"Fireworks AI are industry-leading experts in [machine learning] ML compilers, which is a critical component of optimizing the speed of our model output," said LaForte. "By partnering with them to run our Stable Diffusion 3 API, we can provide the fastest and most reliable enterprise-grade API platform on the market."

Latent Adversary Diffusion Distillation (LADD), Turbo models

The concept of a diffusion model has always been at the heart of Stable Diffusion, hence its name. Stable Diffusion 3 introduces several innovations beyond the diffusion approach used in the first version of Stable Diffusion.

One innovation is the Multimodal Diffusion Transformer (MMDiT) architecture, which brings a transformer to Stable Diffusion for the first time. This allows for much better text comprehension, as well as much better font writing.

Another innovation worth noting relates to the Stable Diffusion Turbo (SD3-Turbo) model, which is intended to be a faster version of Stable Diffusion 3. SD3-Turbo utilizes a new method called Latent Adversarial Diffusion Distillation (LADD), described in the Stable Diffusion 3 Turbo research document.

"Essentially, SD3-Turbo is much faster than SD3, up to 10 times faster, and produces images that are on average almost as good as SD3," says Laforte.

What happens next? A stable associate

As if the new Stable Diffusion model wasn't enough, Stability AI is also providing an early beta of its next big innovation, dubbed Stable Assistant.

The basic idea behind Stable Assistant is not too dissimilar to how OpenAI's ChatGPT Plus chatbot is integrated with DALL-E 3 to generate both text and images.

Laforte revealed that Stable Assistant is a friendly chatbot powered by Stability AI text and image generation technology using Stable Diffusion 3 and Stable LM 2 12B, which was released earlier this month. He added that with its help, users will be able to generate images from conversations, offer knowledgeable responses, help with writing projects and improve content with suitable images.

"Stable Assistant is intended to be a multimodal Stability AI chatbot where all of our models and API services will be available for use without technical knowledge," says Laforte. "Language and image creation are already integrated, and we plan to further develop Stable Assistant's capabilities by adding image editing in the near future and incorporating models from other modalities available to us: video, 3D, audio, and code."