Project Concept
The project explores how the generative neural network Stable Diffusion can translate Gustav Klimt’s unique ornamental style (the golden period) into the genre of contemporary portraiture.
Klimt is one of the most striking representatives of Viennese Modernism, whose works are distinguished by golden backgrounds, curved lines, ornamental patterns, and complex symbolism.
The goal of this project is to train the Stable Diffusion XL model to understand and reproduce these characteristic features, creating new portraits where the artist’s style is organically combined with contemporary imagery, facial features, clothing, and surroundings.
«The Kiss, ” 1907–1908 / „Judith and Holofernes, ” 1901. Gustav Klimt
«Expectation, ” 1905–1909 / „Portrait of a Woman, ” 1916–1917 / „Portrait of Adele Bloch-Bauer I, ” 1909. Gustav Klimt
For training the neural network, a dataset of 30 reproductions of Gustav Klimt’s paintings, dating from his golden period, was compiled. All images were resized to a 512×512 pixel square format and manually processed to preserve key stylistic elements: gold backgrounds, ornamental patterns, fluid lines, and portrait compositions.
Image Preparation
The Learning Process
The training was conducted in Google Colab using an SDXL DreamBooth LoRA notebook. The base model Stable Diffusion XL was selected, utilizing an improved VAE (madebyollin/sdxl-vae-fp16-fix).
Code Snippet
The dataset was loaded locally into the klimt folder. For each image, the metadata.jsonl file specifies a single prompt that the model associates with the style: «photo collage in KLIMT style»
Dataset Upload
Course Code
Following the initial training attempt with automatically generated descriptions (BLIP), the style quality proved low. Fine-tuning was performed using a corrected metadata.jsonl, which allowed the model to properly absorb the key features. The total training time on the T4 GPU was approximately one hour.
Final Images
After training, the model was applied to generate portraits. Prompts were structured using a template: «photo collage in KLIMT style, [description]».
To achieve diversity, various seeds, guidance scales (from 7 to 8.5), and the number of steps (30–40) were utilized.
Close-up portrait in the style of Gustav Klimt’s work
Female Portrait in the Style of Gustav Klimt’s Works
Female portraits in the style of Gustav Klimt’s work
Female portrait in the style of Gustav Klimt’s work
Prompt Examples:
— «photo collage in KLIMT style, close-up of a woman’s face with gold leaf details, abstract floral patterns, high detail, sharp» — «photo collage in KLIMT style, profile portrait of a man with a beard, blue and silver tones, geometric ornaments, moody lighting» — «photo collage in KLIMT style, a couple embracing, golden swirls, red and gold palette, romantic atmosphere, soft focus» — «photo collage in KLIMT style, full body portrait of a dancer in motion, flowing gold ribbons, dynamic composition, bright colors»
Female portraits in the style of Gustav Klimt’s work
Portrait of a Woman in the Style of Gustav Klimt
Portrait of a man in the style of Gustav Klimt
Cat Portraits Gustav Klimt Style Portrait
The final series of images successfully conveyed several key elements characteristic of Klimt’s work:
— Gold backgrounds and ornaments — Most generations feature golden patches, decorative patterns reminiscent of Klimt’s circles, spirals, and mosaic texture.
— Fluid lines and Art Nouveau — The contours of faces, clothing, and decorative elements often possess a curved, «organic» plasticity.
— Portrait composition — The model maintained the centered format common in many of Klimt’s pieces, as well as close-up facial views, which aligns with the original style.
— Color palette — The generated images are dominated by golden, warm tones, occasionally incorporating blue, green, or red accents, which echoes the palette of the Golden Period.
Conclusion
Training a generative model on reproductions of Gustav Klimt’s paintings demonstrated that the neural network is capable of absorbing key stylistic elements of his work (gold backgrounds, ornamentation, fluid lines) and applying them to new portrait subjects. The quality of the resulting output is directly dependent on the diversity of the dataset and the precision of the prompts.
For further refinement, the dataset could be expanded with images from various angles, examples of contemporary portraits could be added, and additional generation control methods (ControlNet) could be employed.
