Stability AI's new SD3 image generator is now available through the API

Decrypt Center for Art, Fashion and Entertainment.

Stability AI has just opened SD3, the fourth iteration of Stable Diffusion's popular text and image generator, to software developers. The announcement of the SD3 application programming interface (API) follows the initial introduction of the model in February 2024, which marked a significant leap in artificial intelligence-generated imagery.

SD3 is the successor to SDXL, continuing a line that includes the very popular SD 1.5 and the very unpopular SD 2. Unlike its predecessors, SD3 users cannot download weights to run models locally or fine-tune them.

The new model represents a change in the Stability AI approach - at least at launch. However, the company said source material may be available later.

"In line with our commitment to openness in generative AI, we intend to make the scale model available for self-hosting with Stability AI membership in the near future," the company said in a statement released earlier today.

Stability AI announced its membership program well before SD3. Non-commercial use is available, but users who want to profit from the model must pay $20 - as long as their annual revenue does not exceed $1 million. For those who earn more, an enterprise version with customized pricing is available.

Method SD3

The SD3 architecture is based on a diffusion transformer that uses two separate sets of weights for text and image embeddings. This approach allows both segments to operate in their own space while considering the other, resulting in improved image generation.

The modernization of Stability AI makes it more competitive with MidJourney and Dall-E 3, if not the current industry star Ideogram. SD3's real potential will be unlocked when the community can refine and adapt it to different needs - such as manga, hyper-realism, and cinematic styles.

That Stability is leaning more towards a membership model is also a shift in approach, providing an additional revenue stream. It's still unclear whether SD3 will be available to non-commercial users. The company did not respond to Decrypt's request for comment.

Stability AI has separately released alternative AI image generator models such as Stable Cascade (based on the Wurschten architecture) and Deepfloyd IF, as well as versions adapted for fast rendering such as Turbo and LCM models.

How to run SD3 via API

Decrypt has put the new model to the test and has come to at least one definite conclusion: It is difficult to use for anyone without at least a little coding knowledge. Also, the tutorials on Stability AI just suck.

So here are simplified instructions provided by DKRacingFan from the MattVidPro AI Discord server, who helped us understand what we were doing.

First of all, it should be noted that SD3 via API requires API credits, meaning you will have to pay. However, you will get free credits if you sign up for Stability's membership programs.

First, go to https://platform.stability.ai and log in with your Google account or register if you are a new user. Then go to https://dreamstudio.ai and create an account using the same email. You will then have free credits in your account that you can purchase.

Then go to https://platform.stability.ai/account/keys and copy your API keys. You don't need to review them, just click the copy button. Remember, as with cryptocurrency startup phrases, never share your API keys or your credits could be charged - or your account could be used to perform prohibited functions.

Next, you will need to install Python. This may take a few minutes.

After installing Python, open a command prompt (search for "cmd" in the Windows search bar) and type the following command:

 pip install requests 

Next, you will need to install Microsoft Visual Studio 2022 if it is not installed on your computer by default.

Then copy and paste the following text into Notepad, Google Docs, or any other word processor to create a text file:

 import requests response = requests.post( f "https://api.stability.ai/v2beta/stable-image/generate/sd3", 
 headers={ "authorization": f "PUT YOUR API KEYS HERE", "accept": "image/*" }, 
 files={ "none": ''}, 
 data={ "prompt": "YOUR PROMPT HERE", "model": "sd3", "aspect_ratio": "16:9" }, ) 
 if response.status_code == 200: with open("./SD3Output.jpeg", 'wb') as file: file.write(response.content) 
 else: raise Exception(str(str(response.json())))
 

Now extract the previously saved API keys and paste them into the text file where it says "PUT YOUR API KEYS HERE".

Then select all the text in the file and copy it. Open Visual Studio and start a new Python project by going to File -> New -> Project -> Python App. Paste the copied text into the left empty field.

Replace the text "YOUR PROMPT HERE" with what you want to generate. Then click on the little "Play" button in the menu to execute the code (or press Ctrl+F5).

A new command window will appear. Wait until it shows that you can click anywhere to exit. The image will then be generated and will appear in the location where you installed the new project. The default is:

C:\Users/YOURUSERNAME/source/reposition/PythonApplication1/PythonApplication1.

That's it! Grab a tissue and wipe away your tears! Repeat these steps with more prompts to create more images. Happy creating!