AI Generation Evaluation
September 9, 2022
Dalle-Mini is a program that allows the user to generate images from a prompt using AI. Artificial intelligence is a topic that is often debated about. Dalle-mini is more of a proof-of-concept demo, and a new version, Dalle-2, is currently in development, but few people have access to it, so the current version will have to do.
In order to use Dalle-mini, someone first needs to select a prompt. Then, the run button is clicked, and around 45 to 90 seconds later, images are generated. Nine images of varying quality and clarity are created. Images will be generated from a list of prompts, and the clearest and most accurate images will be chosen from each group.
The images are rated in two categories. These categories are accuracy and clarity. Accuracy will be judged as follows: the image will be shown to people, and the closer they guess to the prompt, the higher it will be scored. Words that are not entirely correct but are generally close will count for a higher score.
Clarity will be judged by how clear the image is. The less smudging there is, the higher the clarity score will be. The process will be explained to nine total people, and three people will evaluate each image.
The first prompt will be simple: just “pink headphones.” The scores are as follows; accuracy is 90%, clarity is 87%. This is pretty good for this program, as usually it generates something blurry.
The second prompt is “fancy car.” The three people questioned scored an accuracy of 60%. The image was very clear, but the wheels could be better, according to one of them, so it scored a clarity of around 80%. The answers given were “old car,” “vintage car,” and “sports car.”
The third and final prompt was more complicated: “Leaning Tower of Pisa.” Surprisingly, the image was very clear, despite the extra word being added. This is most likely due to there being more reference images for the AI to use. With an accuracy of 100%, and a clarity of about 90%, this is probably the best of the three.
Dalle-Mini is quite the impressive program, and is one of the more prominent of its kind. It can generate beautiful images, if given multiple chances. This program is quite impressive, but it still has a long way to go.