Adobe Inc. brought image-generating artificial intelligence directly into its flagship image editing tool Photoshop in beta as per an announcement today starting with “Generative Fill,” a capability that can modify photos and images with a simple text prompt.

The new feature is powered by Adobe Firefly, a powerful image-creating AI model family. Users interact with it similarly to every other text-to-image generator on the internet, anyone who has made images using OpenAI LP’s DALLE 2, Stable Diffusion or Midjourney will be familiar with entering text prompts and receiving vivid images back, it is very similar.

Using Generative Fill, users can add photorealistic or surreal content to their own images using simple text prompts and extend the edges of their images, all non-destructively. The same feature could be used to remove objects from an image and the AI would fill in the background seamlessy, although a similar feature has existed in Photoshop previously. Firefly uses what’s already in the image to produce the proper way to add what the user asked for and fit it appropriately.

“Generative Fill automatically matches the perspective, lighting, and style of your image to make previously time-consuming tasks swift and delivers results that are truly mind-blowing and delightful,” said Pam Clark, vice president of product management and product strategy for Photoshop.

The examples are endless. A user could take a picture of themselves standing on a beach and extend it to the side, leaving a blank canvas then ask the AI to extend the image, it would then generate the rest of the ocean as if the photo had continued. By selecting a portion of the water and entering a text prompt, boats could be added to the water with perspective correct, or a pod of humpback whales breaching the surface. A more whimsical person could ask the AI to reveal Atlantis rising from the depths.

It could also be used to take a small section of an image and expand it out into a full scene. A single flower in the grass with a prompt: “A flower in a meadow with a blue sky and a forest with mountains.” Would have the AI produce an idyllic scene replete with similar flowers and the mountains and forest as requested, as long as the canvas was large enough to accommodate. The user could then make selections around the newly generated image and add a cabin, more animals or anything else that met their fancy.

The capability is integrated into every selection tool in Photoshop and when it’s used it produces a new “Generative Layer,” which means that it won’t mess up whatever is being worked on. Users can just delete whatever they produced and go about their workflow or merge it into their design at their leisure.

The tool was also designed for commercial users with high-quality assets behind the image generation capabilities, which means that when text prompts call for something – be it a cat, dog, car or unidentified flying object – it will be high resolution and not based on copyrighted content. This means that it’s safe for commercial use in marketing materials or by brands, although Adobe warned that it’s currently a beta product and still experimental so commercial interests might want to avoid using it in their workflows at this time and just test it out currently.

The rise of these tools has also given some pause to critics who fear that generative AI may lead to cause for concern about deepfakes. Especially with the relative ease that features such as Generative Fill provides to simply add or remove elements of a photo and make it look instantly realistic. In very recent news, an AI-generated image of the Pentagon on fire caused a row after a Twitter account claiming to be a Bloomberg news account.

With these issues in mind, Adobe does work to block potential by prohibiting users from creating images that include political figures and will stop images that involve putting “black smoke” in front of the White House or other significant landmarks. Also, its AI usage guidelines explicitly bars users from producing graphic violence, nudity, offensive or fraudulent content and more.

To get on board with testing out the new generative AI capabilities in Photoshop, users need to download the Photoshop beta app. However, they will first also need to be a subscriber to Photoshop or Adobe Creative Cloud, which is available for a seven-day free trial.

Image: Adobe