Vibeus“Krea 2 Fine-Tuning Goes Horribly Right” — u/MellyDArt: So I set... - Vibeus
“Krea 2 Fine-Tuning Goes Horribly Right” — u/MellyDArt:
So I set out to improve Krea 2’s photographic realism. Seven training runs later, I had a checkpoint that produced some extremely convincing photos....and seemed to have forgotten it was ever supposed to do anything else.
A big part of this experiment came from finding an old hard drive containing about 10,000 photographs I’d taken over a decade. A lot of the training data came from my own photography, so this became an opportunity to see what the model could learn from an archive I forgotten I had.
The original goal was to push realism further while keeping the flexibility that made the base model useful. As the photographic results improved, though, it became tempting to keep going.
The process involved seven sequential fine-tuning runs using seven different photographic datasets. These weren’t independent experiments starting fresh from the original base; each run continued from the previously trained checkpoint.
The datasets ranged from roughly 300 to 2,500 photos, with about 20% overlapping data between them. Each run went for a maximum of 4,000 steps. For the final run, I switched from the balanced setting to the style/content-style setting in AI Toolkit.
The 10,000-photo archive + Instagram was a source for the training material, rather than a single dataset I fed through all at once. And that final settings change is part of the history, but I can’t isolate it as the cause of what followed. Several datasets and successive rounds of training were involved, so this is an account of the experiment rather than a controlled comparison.
The photography improved.
By checkpoint 12500, the photographic results were good enough that I didn’t want to dismiss the experiment as a failed fine-tune. It had developed a strong, convincing photographic look that I wanted to preserve.
But once I started testing beyond that, the trade-off became clear. Other styles had become difficult or effectively unavailable. Prompts that should have moved it toward illustration or another visual treatment kept getting pulled back toward photography.
The model had developed a specialty.
LoRA compatibility was another major issue. Every LoRA I tested stopped behaving properly on the checkpoint.
What I was seeing looked consistent with catastrophic forgetting and weights being blown TF out: the model had become much better at the photographic distribution I was training toward while losing capabilities and flexibility from the original base.
The practical outcome was a highly specialized photography checkpoint, with a substantial amount of the original model’s versatility sacrificed along the way.
That left me with a result I couldn’t quite call a failure.
I’m sharing this in case anyone else is experimenting with sequential full fine-tuning or trying to recover a useful look from an overly specialized checkpoint. This isn’t a recipe I’d present as repeatable without further testing, but it seemed worth documenting.
Images: ten generated examples from the experiment, selected from the eleven-image source gallery.
