Build your own vision transformer and diffusion models for text-to-image generation–from scratch!

Build a Text-to-Image Generator (from Scratch) takes you step-by-step through creating your own AI models that can generate images from text. You’ll explore two methods of image generation—vision transformers and diffusion models—and learn vital AI development techniques as you go.

Build a Text-to-Image Generator (from Scratch) teaches you how to:

 • Build and train models to generate high resolution images based on text descriptions
 • Edit an existing image based on text prompts
 • Build and train a model to add captions to images
 • Build and train a vision transformer to classify images
 • Fine-tune LLMs for downstream tasks such as classification, text or image generation
 • Better differentiate real images from deepfakes

Build a Text-to-Image Generator (from Scratch) dives into the powerful models behind AI image generators like DALL-E and Stable Diffusion. We believe that the best way to learn is to build something from scratch, so in this book you’ll build your very own diffusion model and vision transformer. As you work through each stage of development, you’ll develop an understanding of how these models can be customized, applied, and integrated for impressive multimodal AI.

About the book

Build a Text-to-Image Generator (from Scratch) guides you through creating AI models that can generate amazing images from simple text prompts. You’ll explore two distinct methods, learning how transformers turn images into sequences of patches, and how diffusion models refine noise into coherent images. Author Mark Liu explains each stage with clear text, diagrams, and examples. You’ll develop models that can classify images, automatically add image captions, reconstruct images, and deliver high-resolution content. By the time you’re done, you’ll have a deep understanding of how image generation AI works—and the satisfaction of building your text-to-image models!

About the reader

For machine learning enthusiasts and data scientists with intermediate Python skills.

About the author

Mark Liu is the founding director of the Master of Science in Finance program at the University of Kentucky. He is also the author of Learn Generative AI with PyTorch.

Get a free eBook (PDF or ePub) from Manning as well as access to the online liveBook format (and its AI assistant that will answer your questions in any language) when you purchase the print book.

Dr. Mark Liu is a tenured finance professor and the founding director of the Master of Science in Finance program at the University of Kentucky. He has more than 20 years of coding experience, a Ph.D. in finance from Boston College.