Collage image of flowers, eyes, and electronics

How to effectively prompt for AI art and generative AI image creation

SkillsCategory
16 min read
Amy K

Have you ever wondered how to create realistic and stunning images from a few words or a simple sketch? Thanks to the power of generative AI, you can do that and more! One of the most amazing applications of generative AI is image creation. Whether it's crafting images from scratch – like animals, landscapes, faces, artworks, and more – or enhancing and transforming existing images with filters, effects, or styles, the creative potential is limitless.

The secret to harnessing this potential lies in crafting effective prompts. These are your magic words that instruct the AI image generators – be it Midjourney, DALL-E, Stable Diffusion, or any other model – to create desired outputs.

AI images are created by algorithms trained on vast datasets of visual imagery. When you provide these models with a prompt (text, image or both), they "imagine" a new image based on the patterns they've learned, allowing you to generate images and artwork from nothing more than a few descriptive words or a picture. The results can range from abstract to incredibly realistic, opening the door to a wide range of visual content possibilities, for example:

  • Art: Create original, expressive artworks that embody your creativity and vision.
  • Design: Generate tailored assets like logos, icons, illustrations, mockups, and more.
  • Business: Leverage AI for diverse content creation, marketing and advertising, product visualization, brand identity.
  • Education: Simplify complex concepts with custom diagrams, charts, and maps.
  • Entertainment: Design characters, scenes, stories, games, and more to engage your audience.

Disclaimer: When using AI tools, avoid entering sensitive information and always review output for accuracy. Additionally, this content is for demonstration purposes only; it does not represent any affiliation, endorsement or sponsorship with the companies mentioned. All trademark rights belong to their respective owners. Third-party trademarks are used here for demonstrative and educational purposes only, use does not represent affiliation.

So how do you create images using generative AI? Here are the main steps:

1. Choose the AI generative model

The first step is to choose a generative model and signup/login. Each image generator has its own unique strengths and features. Explore them to find the one that best suits your needs. Some are free and many have a free trial, though this has been changing fast. Try them out to see what the images look like and what will work best for you.

Some of the popular and accessible generative models for image creation are DALL-E, Midjourney, and Stable Diffusion (Dream Studio for the online version).

There are an array of generative AI models and tools that can be harnessed for image creation. Different types of generative models can be accessed through various resources and options, tailored to different user experiences:

  • Online platforms: are the most accessible and easiest to use, they offer web-based interfaces and APIs to interact with the generative models. Some examples are Dall-E / OpenAI, Stability, Artbreeder, RunwayML, etc.
  • Libraries and frameworks: For those with coding experience, you can use libraries and frameworks that provide code and documentation to implement and customize the generative models. Some examples are TensorFlow, PyTorch, JAX, etc.
  • Pre-trained models and datasets: You can use pre-trained models and datasets that are available online or offline to run and fine-tune the generative models. Key resources include Hugging Face, Google Colab, Kaggle, etc.

DALL-E 2

Example prompt for AI images using DALL-E 2

DreamStudio

Example prompt for AI images using DreamStudio

Midjourney

Screenshot of example prompt for MidJourney

2. Prepare the input

To get the best results, imagine that you are commissioning an artist or designer or an artist, and prepare to speak to the AI as you would speak to them, describe what you want them to create, and even providing sketches and images.

Depending on the model, you can provide different types of inputs, such as:

  • Text: You can use words, phrases, sentences, or paragraphs to describe what you want the image to look like, these are called prompts. You can also use keywords, modifiers, or references to specify the details and features of the image, and what are called parameters.
  • Sketch: You can use lines, shapes, colors, or symbols to draw what you want the image to look like. You can also use annotations, labels, or hints to indicate the elements and properties of the image.
  • Image: You can use an existing image as the input for the generative model. You can also use masks, crops, or edits to modify or transform the image.

You can use a combination of these inputs as well, for example, sketches and text, or multiple images and text.

I encourage you to always use your own original input.

This is an example where I input my original artwork into DALL-E 2 (image only, without a text prompt). Original on the left, generations on the right: 

Example of Dall-E 2 image only prompt

This is an example where I input my original artwork into DreamStudio plus a text prompt. You can see the original on the left, and four new, similar generations on the right:

Example of DreamStudio image and text prompt

3. Crafting effective text prompts to get the image you want

The secret to getting the most out of AI image creation generation lies in how well you instruct the AI to create what you envision, called prompting. The prompt is your instruction to the AI on what you want it to create. The more focused your prompt is, the closer the AI can come to actualizing your creative conception.

Prompting is an iterative process. Get creative and experiment with different combinations and variations of prompts. Explore various styles, mediums, and settings to discover unique and engaging results. Experiment with different prompts and be open to revising them based on the AI's outputs. Adjust your prompts as needed to achieve the desired results. Fine-tuning your prompts through trial and error will help you discover what works best and come up with creative solutions.

Related: Prompting 101: Writing prompts for AI

Tips to help you craft stellar prompts for compelling visual outputs

Here are some tried-and-true tips on getting the most out of AI art generators:

Compose full sentences with precise and descriptive language

A thoroughly articulated prompt often translates into a more precise and authentic image, so it's worth it to include details about colors, objects, style, etc. The more specific your prompt is, the closer the AI can get to realizing your creative vision.

Longer length and descriptive language

Many models do well with 5-7 word prompts, but some do even better with longer, more thorough prompts. Strike a balance between providing enough details for a compelling image and keeping the prompt concise. Use vivid and concrete language for more predictable results, or experiment with poetic and abstract wording for surprising outcomes.

Use a structured approach for prompts 

Begin with the image content, then move on to the art form, style, and artist references, and finally include additional details such as lighting, colors, and framing. Use commas to separate different elements within your prompt for better clarity and interpretation by the AI model.

Be specific and include relevant details

Provide specific details about the image you envision. The more specific details you provide, the better the AI will understand your vision. Mention specific objects, settings, or actions you want in the image. Specify elements such as subject, composition, colors, and style. For instance, if you own a coffee shop and want an image for social media, you can say, "Generate an image of a cozy coffee shop interior with people enjoying their drinks, using warm and inviting colors."

Utilize descriptive adjectives

Incorporate adjectives that convey the mood or style you desire for the image. For instance, if you want a modern and minimalist design for a website banner, you can say, "Create a sleek and minimalist image with clean lines and a contemporary feel."

Provide examples and incorporate reference images

Including reference images along with your prompt can offer a visual guide for the AI. Upload images that depict the style, composition, or subject matter you want the AI to emulate. You can share similar images (for example stock photos that you own the rights to, or digital images that you created) to provide visual references.

Information to include in your prompt

These are some key points you should consider when entering in your prompt information:

Subject

Clearly state the main focus of the image, which could involve people, landmarks, products, designs, or recognizable entities. For example, a bakery might want an image of a whimsical cake creation or a croissant and coffee in a cozy café setting.

Composition

Describe the arrangement of elements within the image, such as the positioning of objects or subjects. For instance, "a portrait of a woman holding a book on the left side of her office desk."

Color palette

Specify the desired colors to guide the AI in creating the image. You can use terms like "vibrant," "pastel," or "monochrome" to influence the overall color scheme.

Style/Medium

Guide the AI model toward a specific aesthetic approach by mentioning the desired artistic medium or style. This can include paintings, illustrations, 3D renderings, photos, anime/manga, and more. For instance, you might request a "watercolor painting" or a "realistic product photo".

Emotion

Use words that express mood or emotion to guide the AI's interpretation. Consider terms like "joyful," "mysterious," or "serene" to convey the desired emotional tone.

Perspective

Indicate the viewpoint or perspective you want the image to have, such as "bird's-eye view," "close-up," or "wide-angle." You can also specify camera-related terms like "depth of field" to further refine the desired effect.

Lighting

Describe the lighting conditions to set the ambiance of the image. Terms like "brightly lit," "shadowy," "backlit," "dramatic lighting," or "volumetric lighting" can help guide the AI in creating the desired lighting effects.

Texture

Mention the desired textures you want to see in the image, such as "rough," "smooth," or "glossy." This can influence the surface appearance of objects within the generated image.

Time period

If you're aiming for historical or futuristic images, specify the time period or era you want the image to reflect. Examples include "Victorian," "medieval," or "futuristic."

Genre

If you have a particular genre in mind, mention it in your prompt. Whether it's "sci-fi," "fantasy," or "western," indicating the genre can help guide the AI's creative process.

Abstract concepts

AI can handle abstract concepts, so feel free to incorporate them into your prompts. For example, you can request an image that conveys "the feeling of autumn" or "the concept of freedom."

What to omit

To exclude certain elements from the image, clearly state what you don't want to be included. For instance, you could mention "no text," "no logos," or "no people" if they are not relevant to your prompt. This may be done under a parameter setting.

Parameters

Each AI model offers unique settings, called parameters, that help fine-tune the desired outcomes. These include specifying aspects like the ratio (e.g., "3:4" or "16:9"), excluding certain elements, or adjusting the level of variation. The available parameters vary with different AI tools, so consult the specific documentation of the tool you're using to understand its unique capabilities. Experiment and use a process of trial and error with parameters to help you discover both more control and new creative possibilities and unlock the full potential of the AI image creation generator's capabilities.

Here is the same prompt used across different generators, with a “no text” parameter:

DALL-E 2

DALL-E 2 prompt with no text parameter

DreamStudio

DreamStudio prompt with no text parameter

Midjourney

MidJourney prompt with no text parameter

The prompt I used for each was: 

“business logo for a car detailing company in gold linework on a navy blue background. showing a retro 50s-60s vintage sports car grille in a circle with classic curves, captured in a moody, low key light with selective focus on the grille, classic, retro, moody, simple, elegant. --no text”

Tips and best practices

  • Embrace flexibility and iterate: Be prepared to experiment with varying inputs and outputs to fully understand the capabilities and constraints of the generative model.
  • Foster originality: Allow your imagination to run free, crafting inputs that are a reflection of your unique style and personality.
  • Keep trying, patience is key: The process may involve a good deal of trial and error. Persevere, refine your inputs, and anticipate breakthroughs.
  • Understand your tools: keep in mind that each AI model has its own unique features and capabilities. Consult the documentation of the specific tool you're using to fully understand the parameters and controls at your disposal.
  • Explore and experiment: Embrace the process of exploring and experimenting with different elements to uncover fascinating and visually captivating results.

Where to find inspiration and prompt ideas

Crafting effective AI art prompts is both an art and a skill that develops over time. When you're starting out with prompts, visualizing the exact results you desire can be challenging. Here are some ideas to get you started:

Community engagement

Draw inspiration from the experiences of other creators in the community. Join Discord servers like OpenAI's and Midjourney's to engage in discussions on AI image creation generation. Participate, ask questions, and collaborate with fellow creators to gain insights and discover new possibilities.

Curated collections

Explore curated collections of images and corresponding prompts on platforms like neural.love's public library or websites like Lexica. These resources offer a wealth of inspiration, allowing you to explore original descriptions and discover fresh combinations of details that may spark ideas.

Industry references

Look for inspiration within your industry or related fields. Research relevant artists, design styles, or visual trends that align with your business's theme. However, be cautious about copyright and infringement issues, and strive to develop your own distinct style.

Business focus

Tailor your prompts to reflect your small business's identity, products, or services. Consider elements that showcase your unique offerings and create AI image creations that resonate with your target audience.

Visualize brand values

Use prompts to convey your brand's values and personality. Consider how colors, styles, and compositions can evoke emotions or align with your brand identity.

Customer engagement

Engage with your customers or followers to gather prompt ideas. Conduct surveys, polls, or social media interactions to gather insights into their visual preferences. Incorporate their feedback into your prompts to create more personalized and relatable images.

Make like an artist and embrace the creative journey: Crafting effective AI art prompts is both an art and a skill that develops over time. Embrace the process of exploring and experimenting with different elements to uncover fascinating and visually captivating results. Allow AI to assist you in bringing your creative vision to life, opening up new possibilities and sparking inspiration along the way. Enjoy the journey of discovery and let the AI-powered algorithms amplify your artistic endeavors!

Advanced tools and strategies

As you delve deeper into the world of AI-generated images, there are a range of advanced tools and strategies to take your prompts to the next level. Here are some exciting options to help you elevate your prompt engineering skills and unleash your creativity:

Image-to-image

Explore image-to-image generation as an alternative to text-to-image prompts. Use an existing image as inspiration and further customize it with text or model controls. Platforms like Midourney and Runway provide this capability, allowing you to experiment with different artistic effects and modifications. 

Reverse engineer real images

Input an image to get a description. This can be done in Midjourney, or leverage tools like the CLIP Interrogator hosted at Hugging Face. This AI model takes an image as input and generates a text prompt based on its content. You can use the result as a starting point to build more detailed prompts for text-to-image models. For example, if you're a bakery owner, you can reverse engineer a picture of a stunning cake and use the generated text prompt to refine your desired cake design further.

AI-powered image editing

Leverage AI-powered image editing tools like Canva or Photoshop to manipulate images based on text prompts. Select specific parts of an image and use prompts to guide AI algorithms in making targeted changes. This technique opens up possibilities for refining compositions, adjusting colors, or adding visual elements to align with your creative vision.

AI-generated prompts with ChatGPT or other chat bots

Explore the synergy between AI models by using ChatGPT to generate text prompts for AI image creation generation. By engaging with an AI chatbot, you can unlock new aesthetics, compositions, or subject combinations that might not have crossed your mind initially. This collaborative approach can spark fresh ideas and push the boundaries of your creative explorations.

Train your own AI creation image model

Consider tools like Leap AI that allow you to train your own image model. This empowers you to create a customized model that aligns precisely with your small business's unique style and preferences. By reducing the variance in outputs that you might encounter with "public" image generation models, you can ensure a consistent and distinct visual identity for your brand.

Considerations for usage, rights and ethics in AI image generation

Before using AI-generated images, especially for your business, be sure to first check current regulations and always be sure to always read and understand the platform/app’s legal terms. Commercial use rights can vary based on the platform's terms of service and user agreement and change regularly. Always check with the platform for the most current information.

When using images as inputs, be sure to respect copyright restrictions. 

There may also be legal and ethical issues surrounding rights and usage. Some data sets have used images without the artists or creators' consent, which could be a major copyright or infringement issue and is unethical as the original creators have not agreed to nor been compensated for use of the work or style. Seek ethical datasets that have rights and permissions. If you are an artist, designer, photographer, or image creator, you can search if your work has been used to train datasets at https://haveibeentrained.com/ and may select to opt out (or opt in) there. 

AI image creation generation can present challenges, including biases, prompt complexity, and dependency on the quality of data. Inadequate or skewed data may produce restricted or even detrimental images. Be mindful that AI-generated imagery can manipulate perception, although most platforms have preventative measures.

The ultimate goal of using AI is to generate unique images, not to reproduce or resell the work of others.

Conclusion

Generative AI image creation tools are a potent resource that brings unprecedented value to your fingertips. Whether you want to generate expressive artworks, design tailored assets, simplify complex concepts, or engage your audience with captivating visuals, generative AI can be a powerful tool in your creative arsenal and open a world of possibilities for you. So, embrace the creative journey, experiment with AI art prompts, and bring your imaginations to life.