{"cells":[{"cell_type":"markdown","metadata":{"id":"WqcPzKEDZigO"},"source":["# Introduction to Stable Diffusion Using Google Colab Workshop\n","Jim Plaxco, www.artsnova.com\n","\n","\n","---\n","\n","#SECTION 1: STABLE DIFFUSION\n","\n","---\n"]},{"cell_type":"markdown","metadata":{"id":"fDNbJ0xzZsbL"},"source":["**Step 1**: Make sure that the runtime type is GPU \n","        ( Runtime -> Change Runtime type -> Hardware Accelerator =  GPU )\n","        Click Save after selecting GPU\n","Verify GPU availability\n","\n","Documentation for the nvidia command is at https://developer.nvidia.com/nvidia-system-management-interface"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"jYDa7v87aANF"},"outputs":[],"source":["!nvidia-smi"]},{"cell_type":"markdown","metadata":{"id":"mDTPwXe5Z1hh"},"source":["**Step 2**: Install the various libraries. Note that some libraries may already be installed"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"6w7YgzlPTEWE"},"outputs":[],"source":["# install the libraries using pip3 (Package Installer for Python 3)\n","# NOTE: accelerate is not required but speeds up the inference process a lot \n","!pip3 install diffusers transformers scipy ftfy accelerate\n"]},{"cell_type":"markdown","metadata":{"id":"Q6mhfHzHaFkj"},"source":["**Step 3**: Once libraries are installed, Import the various libraries."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"mscmI-PYTNpG"},"outputs":[],"source":["#  import the various libraries needed by Stable Diffusion\n","from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler, DiffusionPipeline, DPMSolverMultistepScheduler\n","import torch\n","\n","# the following libraries are used to output the image to the screen\n","from PIL import Image\n","from IPython.display import display"]},{"cell_type":"markdown","metadata":{"id":"TmEipsasaSJE"},"source":["**Step 4**: Import the Hugginface interface so you can enter your Stable Diffusion API key (token) using the notebook_login() function\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"v2IdBeciUTEm"},"outputs":[],"source":["#@title Login\n","# have your Huggingface API key (token) ready\n","from huggingface_hub import notebook_login\n","notebook_login()\n"]},{"cell_type":"markdown","metadata":{"id":"_pGaZA9xajjn"},"source":["**Step 5**: Set up the processing pipeline. And while we are at it, disable the NSFW (Not Safe For Work) filter. \n","\n","Note that if Stable Diffusion thinks the output image may be NSFW, you will get an all black image as output. "]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5iokTcsiTTOp"},"outputs":[],"source":["# create the  pipeline (this pipeline is made with Euler Scheduler):\n","model_id = \"mann-e/mann-e_rev-3\"\n","\n","scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder=\"scheduler\")\n","\n","pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, torch_dtype=torch.float16)\n","# use GPU\n","pipe = pipe.to(\"cuda\")\n","\n","# disable NSFW filter \n","# the following code is optional and may be deleted\n","def nonsfw(images, **kwargs): \n","    return images, False \n","\n","pipe.safety_checker = nonsfw\n"]},{"cell_type":"markdown","metadata":{"id":"Eca7n9l5eOul"},"source":["**Step 6:** Generate Images.\n","This section will be run each time we want to create a new image. It is the only code block that you will run more than once.\n","\n","**Google Colab Parameters**\n","#@param is a parameter for the Colab GUI that allows you to set variable values using the Colab GUI instead of inserting them directly into the code.\n","\n","**Stable Diffusion Parameters**\n","\n","**prompt**: This is the text prompt that describes what you want your image to look like\n","\n","**negative_prompt**: This is what you do not want to see in your image\n","\n","**inference_steps**: The number of steps of processing. This value impacts how long it will take to generate an image. A good starting value is 50\n","\n","**guidance**: How strictly or losely do you want your words to be interpreted. A good starting value is 7.5 \n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"TC-WD6gxTWLs"},"outputs":[],"source":["# start the image generation process\n","# Set your parameters\n","prompt = \"vintage rocket flying in space, concept art, centered, hdr render\" #@param {type:\"string\"}\n","negative_prompt = \"low quality, blurry\"  #@param {type:\"string\"}\n","inference_steps = 28  #@param {type:\"slider\", min:0, max:240, step:1}\n","guidance = 7.0    #@param {type:\"slider\", min:0, max:20, step:0.1}\n","width = 768      # width in pixels must be a multiple of 256\n","height = 512     # height in pixels must be a multiple of 256\n","\n","# Generate the image\n","image = pipe(prompt=prompt, \n","             negative_prompt=negative_prompt, \n","             num_inference_steps=inference_steps, \n","             width=width, height=height, \n","             guidance_scale=guidance).images[0]\n","#image.save(\"My_image.png\")\n","filename = prompt.replace(\",\",\"\") + \"-Steps\"+str(inference_steps) + \"-guide\"+str(guidance)+\".png\"\n","filenameout = filename.replace(\" \", \"_\" )\n","print(filenameout)\n","image"]},{"cell_type":"markdown","metadata":{"id":"FY5S7BcShMtF"},"source":["\n","---\n","A SECOND VERSION OF THE CODE IN STEP 6 EXECPT IT PRINTS OUT A FILENAME THAT CAN BE USED WHEN SAVING A FILE. \n","\n","---\n","\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"anq2z8sdQIs9"},"outputs":[],"source":["# start the image generation process\n","# Set your parameters\n","prompt = \"waterfall with swirling water in a shallow wide stream, hdr, colorful, detailed\" #@param {type:\"string\"}\n","negative_prompt = \"low quality, blurry\"  #@param {type:\"string\"}\n","inference_steps = 8 #@param {type:\"slider\", min:0, max:240, step:1}\n","guidance = 2.3  #@param {type:\"slider\", min:0, max:20, step:0.1}\n","width = 768 \n","height = 512 \n","\n","# Generate the image\n","image = pipe(prompt=prompt, \n","             negative_prompt=negative_prompt, \n","             num_inference_steps=inference_steps, \n","             width=width, height=height, \n","             guidance_scale=guidance).images[0]\n","#image.save(\"My_image.png\")\n","filename = prompt + \"-\"+str(inference_steps) + \"-\"+str(guidance)\n","print(filename)\n","image"]},{"cell_type":"markdown","metadata":{"id":"WsLOmy_Bi_Lt"},"source":["Step 7: One way to generate multiple images is to pass a list into the pipeline. This requires more VRAM.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"RLG9ky4RjEbC"},"outputs":[],"source":["# start the image generation process\n","# Set your parameters\n","quantity = 3 #@param {type:\"integer\"}\n","prompt = \"moon in sky polychrome woodblock print\" #@param {type:\"string\"}\n","negative_prompt = \"low quality, blurry\"  #@param {type:\"string\"}\n","inference_steps = 50 #@param {type:\"slider\", min:0, max:240, step:1}\n","guidance = 5.3  #@param {type:\"slider\", min:0, max:20, step:0.1}\n","width = 512\n","height = 512 \n","\n","# Generate the image\n","promptlist = [prompt] * quantity\n","negpromptlist = [negative_prompt] * quantity\n","# image is a Python list with one element for each image\n","image = pipe(prompt=promptlist, \n","             negative_prompt=negpromptlist, \n","             num_inference_steps=inference_steps, \n","             width=width, height=height, \n","             guidance_scale=guidance).images\n","\n","for img in image:\n","  display(img)\n","  print(\"-----\")\n"]},{"cell_type":"markdown","metadata":{"id":"4F92KFS7rUa9"},"source":["Step 8: A second way to generate multiple images is to invoke the pipeline once per image count. "]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"ZbFuMnLTnwwx"},"outputs":[],"source":["# start the image generation process\n","# Set your parameters\n","quantity = 4 #@param {type:\"integer\"}\n","prompt = \"rocketship in orbit around moon, 1950s, detailed  panoramic illustration\" #@param {type:\"string\"}\n","negative_prompt = \"low quality, blurry, words\"  #@param {type:\"string\"}\n","inference_steps = 36 #@param {type:\"slider\", min:0, max:240, step:1}\n","guidance = 6  #@param {type:\"slider\", min:0, max:20, step:0.1}\n","width = 1024\n","height = 768\n","\n","filename = prompt.replace(\",\",\"\") + \"-Steps\"+str(inference_steps) + \"-guide\"+str(guidance)+\"-1.png\"\n","filenameout = filename.replace(\" \", \"_\" )\n","print(filenameout)\n","\n","for indx in range(0, quantity):\n","    # Generate the image\n","    image = pipe(prompt=prompt, \n","             negative_prompt=negative_prompt, \n","             num_inference_steps=inference_steps, \n","             width=width, height=height, \n","             guidance_scale=guidance).images[0]\n","    #image.save(\"My_image.png\")\n","    display(image)\n","# done"]},{"cell_type":"markdown","metadata":{"id":"IL3ZIV5On85W"},"source":["# **The End**\n","\n","NOTE: If at some point your pipeline abends, you should run the following code to reclaim allocated memory"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5DUXNLomJrmu"},"outputs":[],"source":["### USE THIS TO FREE MEMORY AFTER AN OUTOFMEMORYERROR ERROR OR AN ABEND\n","import gc\n","torch.cuda.empty_cache()\n","gc.collect()"]}],"metadata":{"accelerator":"GPU","colab":{"provenance":[{"file_id":"1ZvYrRTol7QY6JKtEtl0iZlDEE-tJJddR","timestamp":1683511112299}],"authorship_tag":"ABX9TyM1wpWP3hLTVNMKm74Gtv1t"},"gpuClass":"standard","kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}