There are a large number of different models for creating photorealistic images in various artistic styles. Some models work better for creating anime images, while others work better for imitating famous artists.
To continue working, we need to follow the following links and download the latest versions of the model files with which we will work:
Realistic Vision версия 5.1
(https://civitai.com/models/4201/realistic-vision-v51);
DreamShaper версия 8.0
(https://civitai.com/models/4384/dreamshaper);
Color Fusion версия 1.0
(https://civitai.com/models/20617/color-fusion);
Midjourney версия 4.0
(https://huggingface.co/prompthero/openjourney/blob/main/mdjrny-v4.ckpt);
Openjourney версия 4.0 (модель обучена на рисунках Midjourney)
(https://civitai.com/models/86?modelVersionId=27392);
SDXL Yamers Realism! 2.0
(https://civitai.com/models/136669/sdxl-yamers-realism-realisticanime3d).
For example, we need to download the Realistic Vision model file version 5.1.
Follow the link above to the page where there is a model file:
https://civitai.com/models/4201/realistic-vision-v51
Then we find the file of the latest version of the model for downloading:
And download the file by clicking the «Download» button:
The realisticVisionV51_v51VAE.safetensors file, 1.9 GB in size, will be downloaded to your computer.
If you do not have an electronic version of the book, you can enter the names of the model files in any search engine and find them on the Internet. For example, upon request «download model Realistic Vision» you will be offered a list of sites for downloading, including the site civitai.com, from which I downloaded most of the files I needed.
After you download all or most of the models offered to you for download, you need to transfer all the files (using the «cut» and «put» commands) to a folder along one of the paths: «C:\EasyDiffusion\models\Stable-diffusion» or «D:\EasyDiffusion\models\Stable-diffusion», depending on where you installed your program.
For this:
Open the folder on the disk where Easy Diffusion is installed (for example, «C:\EasyDiffusion\ or «D:\EasyDiffusion\»).
Then find and open the «models» folder.
In it you will see a folder called «Stable-diffusion». Open it and transfer the model files you downloaded into it.
Congratulations!
You have become the proud owner of your own Easy Diffusion Artificial Intelligence system, installed on your personal computer and supplemented with new models.
Now we can proceed to the first launch.
Step 3. Launch Easy Diffusion for the first time
If immediately after installing the program you selected the «Run Easy Diffusion 3.0» command, as shown in the figure below:
A black window will appear in front of you with running lines of incomprehensible text:
You shouldnt be afraid of this window, since this is the window where the application itself works, or, more precisely, its server part. This window will always appear when the program starts working. There is no need to close it until you decide to finish your work. Just roll it up.
You need to wait a minute or two (depending on the performance (power) of your computer), after which another window will open in the Internet browser (installed by default). This will be the same Easy Diffusion user interface (or it can also be called the client application of the program), with which we will work throughout the entire time.
Easy Diffusions interface is dark, but thats fixable. In the next chapter we will learn how to make our work more comfortable using program settings.
Thus, when we first launched the program, two windows opened: a window for the server part of the program and a window for the interface.
But dont rush to get started.
I would recommend that you close both windows and start the program again. This is primarily due to the fact that when you restart the Easy Diffusion program will automatically check for updates, download and install all the necessary new files on its own.
Wait a little longer and your default Internet browser will open again.
After which we can begin with you the next step of work setting up comfortable and stable operation of Easy Diffusion.
Step 4. Setting up Easy Diffusion
Setting up Easy Diffusion is actually very simple.
By default, the program has options that provide a high level of comfortable work and do not require any changes. However, there are a few that require our attention.
The figure below shows the user interface of the program:
So, lets get started.
Easy Diffusion has only five main application work window tabs, which are placed one after the other at the top of the user interface work area:
«Generate».
«Settings».
«Help & Community».
«Models tools».
«Whats new.»
All our work will take place in the main «Generate» window, which we will devote most of our time to in the next chapter.
In this chapter you will look at the contents of the following bookmarks:
«Settings».
«Help & Community».
«Models tools».
«Whats new».
«Settings»
To get started, we first need the Settings window.
For your convenience, I will use the «Light» theme of the program:
The Settings window provides the following options:
«Theme». Allows you to change the current theme to the one you like. Im using the «Light» theme.
«Auto-Save Images». Lets activate the option. It allows us to save all our images and descriptions for them in a folder we specify.
«Save Location». Leave the default value or change it to the path where our files will be stored. For example, C:\EasyDiffusion\models\Stable Diffusion UI.
«Metadata format». We change to «txt» the text data format in which our requests will be stored. They can be read using any text editor.
«Block NSFW images.» Lets activate the option. Prohibits working with photos of certain categories.
«Enable Sound». Lets activate the option. We will always receive a sound notification that our image is ready.
«Process newest jobs first.» Determines the order in which images are processed. By default, jobs are processed in order. We dont change.
«Extract LoRA tags from the prompt.» LoRA stands for Low-rank adaptation and is translated as «Low-rank adaptation». LoRA speeds up the process of training models. We do not change the parameter.
«Open browser on startup.» The option is responsible for launching the Internet browser after the server part has started. We dont change.
«GPU Memory Usage». The option transfers all program work from the video card to the central processor. If you select «High», the program may run very slowly or stop working altogether. We need to select «Law».
If your video card is not NVIDIA GeForce RTX, then set it to «Low».
«Use CPU (not GPU).» The option shifts all the work of the program not to the video card, but to the central processor. If you select a central processor, the program can run very, very slowly. We dont change.
«Auto-Save Settings». Click the «Configure» button. All options must be selected:
To do this, change the «Enter Prompt» value.
And close the window.
«Confirm dangerous actions.» The option is responsible for the security of the application and must be active.
«Profile Name». Leave the default value «default».
«Beta channel». The option provides access to not yet very tested and unstable functions that may appear in the next version of the application. We dont change.
«Use the new v3 engine (diffusers).» A very important function for us, which in the left panel of Text-to-image AI software increases the list of parameters for customizing the created image. Lets activate the option.
After you have made the changes, click the «Save» button to save all the changes made.
Below are a few more options that you do not need to change:
Three options control the network settings of your Easy Diffusion application. We dont change them.
Please note that under the settings there is system information that allows us to see the data of the CPU and video card we are using.
My computer details are:
Intel64 Family 6 Model 60 Stepping 3, GenuineIntel.
NVIDIA GeForce GTX 980 (cuda:0) (3.5Gb free / 4.3 Gb total).
My laptop details:
Intel64 Family 6 Model 158 Stepping 10, GenuineIntel.
GeForce GTX 1650 (cuda:0) (2.2Gb free / 4.3 Gb total).
And below is the address where we can access our server. In order to open the Easy Diffusion user interface in the Internet browser, write: localhost:9000/ or 127.0.0.1:9000/. After which the programs working window will open.
«Help & Community»
Following the Settings window is the Help & Community window.
In this window, by clicking on one of the links, you can independently study additional information about working with the Easy Diffusion program, as well as learn about installing additional models and modules (plugins).
You can also join the system user community.
From the latest link «Source code on GitHub» you can download the Easy Diffusion source codes from GitHub.
«Models tools» / «Text-to-image AI model software»
Behind the «Help & Community» window is the «Models tools» window, which has two tabs:
«LoRA Keywords».
«Merge models».
In the LoRA Keywords window, you can create sets of keywords for yourself to speed up the image creation process. We will dwell on this option in more detail as we work.
Using the «Merge models» window, you can combine different models with each other. For example, those that we downloaded after installation from the Internet.
If you like a couple of models that you work with most often, you can combine them into one model and give it your name. But there is one small limitation: the models must be of the same type.
From my experience I will say that there is no particular need to combine models. But, if you use two dozen different models, you may need this functionality of the Easy Diffusion application to combine according to the desired topics or styles.
«Whats new»
Following the «Models tools» window is the «Whats now» window. This window provides a list of all updates and changes that have been made to the current and previous versions of the Easy Diffusion application.
It will also be useful for us to know that our Easy Diffusion 3.0 program is 40% faster than Easy Diffusion v2.5, which has a serious impact on the speed of creating our images.
Step 5. Lets get started
As we have already seen, the Easy Diffusion Artificial Intelligence system is a fairly simple program both to install and configure.
Once again, I would like to draw your attention to the fact that the main purpose of Easy Diffusion is to create a new unique image from a text description or from a combination of text and image based on the most popular neural network models today (which we downloaded and installed earlier).
In order for us to create our first image, we launch the program and go to its work area (the «Generate» window):
In the «Generate» window on the left you will see a toolbar that allows you to enter a text description and define all the necessary parameters for the new image you are creating:
For our first image, we will only add a description and select a model to base this image on. We will not change any parameters now.
Please also note that above the Toolbar there are three additional buttons that allow us to add a description from a text or structured file, as well as use additional image modifiers:
«Load from a file». This button allows you to load text describing a previously created image from a text file or a JSON file. It is very useful when you are working with a large number of images. Previously, in the Settings window, we enabled the «Auto-Save Images» option and defined the «Metadata format» as the text «txt». This way our system saves all your images and descriptions in text format automatically.
«Embedding». Configured through the «Models tools» tab of the applications working window.
«Image Modifiers» / «Image modifiers». A very useful button, especially at first, when you have not yet learned how to compose complex descriptions of new images. In the modifiers you can choose several different filters for your images:
Below the buttons there are two windows:
«Enter Prompt». This is the main window, which is intended to contain a text description (we will call it briefly a «query») of the image that we want to create. We will work with him throughout our reading of the book.
«Negative Prompt». This is an additional window in which everything that should not be displayed in the created picture is described in short phrases.
Below the request windows there are two buttons, united by one heading «Initial Image (img2img)»:
«Browse». This button allows us to add our image, which the Easy Diffusion system uses to create a new image similar in design style and presentation form from it and the request we wrote.
«Draw». The button opens the image editing window. In this window we can edit the image we added as a source. For example, clear the background of an image.
Below is the «Make image» button, which is very important for us, with which we will start the process of creating our new unique images.
Under the «Make image» button there is a large list of parameters for the image we want to create.
Lets take a closer look at all the «Image Settings» functions:
«Seed». This is the starting number of the random number generator that is used to create a unique image. By default, the program itself creates a unique number. If we know the Seed and other things being equal, we can create the same picture again. We do not change this option and leave it as it is.
«Number of Images». This option determines the number of images that are created after clicking the «Make image» button. We leave the default value (equal to «1»). If you change the number, it will affect the overall performance of the system. We do not change this option and leave it as it is.
«Model». This window displays a list of models that we downloaded for you earlier. Models are used to create unique images. We will look at their work later in this and subsequent chapters.
«Clip Skip». This option allows the system to more accurately analyze and interpret our request. When the option is disabled, then the new image created matches our request as closely as possible. We do not change this option and leave it disabled.
«ControlNet Image». This is a new and very interesting option of the Easy Diffusion Artificial Intelligence system version 3.0. With ControlNet, you and I can not only create a unique image, but we can also make it look like a celebrity. ControlNet will be covered in more detail in a separate chapter.
«Custom VAE». This option allows us to use a special autoencoder, which helps speed up the creation process and improve image quality. The Easy Diffusion manual suggests downloading and using the file vae-ft-mse-840000-ema-pruned.ckpt. I must say that it worked great with Easy Diffusion 2.5, but does not work correctly with our version of Easy Diffusion version 3.0. We do not change this option and leave it disabled.
«Sampler». These are auxiliary Text-to-image AI software for improving the created image. The interesting thing is that some samplers are suitable for certain types of images, while other samplers are suitable for other types of images. You will have to determine which one is better during your work.