Generative AI means AI models that create new content: text, images, video, audio or program code. They have learned from very large numbers of examples how such content is built, and on request they assemble something new that resembles those examples. A classic AI, by contrast, rates or sorts what already exists.
Picture someone who has read thousands of cookbooks. A filter would only sort recipes into “dessert” or “main course”. A generative AI writes you a new recipe that sounds like it came from one of those books. You ask ChatGPT for a recipe with pumpkin and feta, and out comes a text that did not exist before.
1. Generative or classic: the difference
Many AI systems you met in the past were classic models. They are shown something and return a verdict. Generative models turn the direction around: they receive an instruction and deliver the content itself. More on the broader picture in the entry on artificial intelligence.
- Classic AI
- Detects whether an email is spam
- Generative AI
- Writes a reply to the email
- Classic AI
- Detects whether a photo shows a dog
- Generative AI
- Creates an image of a dog from a description
- Classic AI
- Turns spoken language into text
- Generative AI
- Reads a text aloud in a natural voice
- Classic AI
- Finds known error patterns in a program
- Generative AI
- Writes a new function from your description
gradually.aiThe line is not sharp. A generative model can also rate texts, and many systems combine both. What matters is what comes out at the end: a verdict or new content.
2. How generative AI works
During training, a model sees huge amounts of examples, such as texts, images or recordings. It mostly learns patterns rather than individual examples word for word, such as which words follow each other, how edges and colors fit together in a photo, how a melody unfolds. After training, it holds a kind of probability map of what fits an instruction.
When generating, it draws from that map. A language model calculates a probability for the next token and picks one. Then it repeats this for the one after, until the answer is complete. Because a dice roll is involved, the same question can produce two different answers. How strongly chance plays a role is controlled by settings such as temperature and top-p.
There is a second building block: for a model to compute with text, images or sound, everything is translated into numbers. The entry on embeddings explains the principle behind it. Technically, all of these models belong to deep learning.
2.1 Transformers and language models
The architecture behind most language models is called the transformer. It was introduced in 2017 in the paper “Attention Is All You Need”. Its core piece is an attention mechanism that works out, for each word, which other words in the text matter. A transformer produces content piece by piece. That works especially well for text but is also used for images and audio. The family of large language models has its own entry. According to its 2020 paper, GPT-3 already had 175 billion parameters.
2.2 Diffusion models
Many current image generators work with diffusion. During training, an image is covered with noise step by step, and the model learns to reverse that step. When generating, it starts from pure noise and removes it in many small steps, guided by your description, until an image remains. In 2020, Ho, Jain and Abbeel showed that this can produce high-quality images. The latent diffusion approach from 2021 moves the process into a compressed space and cuts the computing effort considerably.
2.3 GANs
A generative adversarial network consists of two networks. The generator creates images, and the discriminator tries to tell them apart from real ones. Both train against each other until the fakes hardly stand out. The idea comes from Ian Goodfellow and colleagues in 2014 and shaped the first wave of realistic AI images. In current image generators, diffusion models have become more common.
2.4 The milestones at a glance
The timeline shows how the field built up:
- Milestone
- GANs (Goodfellow et al.)
- Significance
- Two networks train against each other and produce realistic images
- Milestone
- Transformer (“Attention Is All You Need”)
- Significance
- The architecture today’s language models are built on
- Milestone
- GPT-3
- Significance
- A language model with 175 billion parameters solves tasks from just a few examples in the prompt
- Milestone
- Denoising Diffusion Probabilistic Models
- Significance
- Diffusion produces high-quality images
- Milestone
- DALL·E (Ramesh et al.)
- Significance
- A transformer creates images from text descriptions
- Milestone
- Latent Diffusion Models
- Significance
- Image generation in a compressed space with far less computing effort
- Milestone
- ChatGPT
- Significance
- OpenAI releases a chatbot that answers follow-up questions in dialogue
gradually.ai3. Worked example: one need, three outputs
The same topic can be handled with different model types. An invented example: you want to descale your coffee machine and need instructions, an icon for it and an automatic reminder.
With a language model, as text:
Prompt:
Explain in three short steps how to descale my coffee machine.
Example output:
1. Add descaler to the water tank as directed on the package.
2. Run the mixture through without coffee grounds.
3. Rinse the tank with clear water and run it through twice.With an image model, as a picture:
Prompt:
Flat pictogram of a coffee machine with a water drop, white on a blue background.
Output:
An image file (for example PNG). Not text, but pixels.And with a model for code:
# Prompt:
# Write a Python function that reminds me to descale
# after 60 brewing cycles.
# Example output:
def check_descaling(brew_cycles: int) -> str:
if brew_cycles >= 60:
return "Time to descale."
return "All good."The outputs are illustrations. Ask the same model again and you get a different wording, a different order or a different image. That is not a bug but part of the principle: each time, a new draw is made from the probability map. If the answer does not suit you, a more precise prompt helps more than asking again and again.
4. Which models exist
Our AI model directory lists 808 model profiles across the four main output types. The table shows how they are distributed:
- Profiles in the directory
- 473
- Compare
- LLM comparison
- Profiles in the directory
- 144
- Compare
- Image model comparison
- Profiles in the directory
- 103
- Compare
- Video model comparison
- Profiles in the directory
- 88
- Compare
- Audio model comparison
gradually.aiYou can find all profiles in the AI model directory.
5. Limits and misconceptions
- “The AI knows it.” A generative model predicts what sounds plausible, not what is true. That is why hallucinations happen: invented sources, wrong numbers, smoothly worded errors. Check facts you reuse.
- “The model looks things up.” On its own, it does not. Knowledge from training has a cutoff, see knowledge cutoff date. It only gets access to current or private sources through add-ons like RAG.
- “It is all real.” Generated images, voices and videos can imitate real recordings convincingly. The EU AI Act therefore requires providers to mark the outputs of generative systems in a machine-readable way as artificially generated, and deployers to disclose deepfakes. This is set out in Article 50. More on the framework in the entry on AI governance.
- “Generative AI is an agent.” A model that answers does not act. Only an AI agent uses it in a loop with tools to reach a goal.
- “The result automatically belongs to me.” Whether and how generated content is protected, and how the training data is treated, depends on the country and the individual case. For business use, it is worth reading the provider’s terms of use.
6. What does this mean for you?
- Use generative AI as a drafting tool. It is strong at first drafts, variants and rewording. Responsibility for the result stays with you.
- Choose the model type by the task. Text needs a language model, a logo an image model. The directory has comparisons for every type.
- Give precise instructions. The clearer the task, format and example in the prompt, the more useful the output.
- Check before you publish. Especially numbers, quotes and sources. And label AI content where it is required or where readers could otherwise be misled.
- Start with the free plan. If you keep hitting its limits, take a look at the paid plans. We collect current figures on adoption in the AI statistics.