There is an illusion in the idea of up-front design in software (that is, code arrangement, not algorithm), that makes it seem more plausible than it really is.
When working on it, we have a feeling that if we devise a particularly good structure, it will accommodate more changes. And further, that better techniques – alternatives to classes, templates, or other structures – can be invented to help software development handle change. But there is something not entirely right here.
It is not because the techniques are immature. Better techniques of abstraction (classes, templates, etc.) can never fundamentally help. What you are trying to do in up-front design is to make an abstraction: but one that covers a set whose instances are at different future times. (An abstraction just fixes something, and lets other things vary. In designing one, you distill a generalisation, a simplification, of a set of things.) But that cannot be done if the set is unknown – the commonality of an unbounded set is unbounded. And that is the situation with up-front design and its future instances.
It feels like only the instances are unknown, but it is really the abstraction that would encompass the instances that is unknown.
The only solution is to find the bounds – not make a good prediction, but find with certainty – to make them known. It is only definite fixed bounds that can give a definite fixed enduring up-front abstraction. And when, as usual, that cannot be done, there should be no sense of shortfall. You make the best use of the information available, and when that changes, the design should too. Software development is essentially about responding to change, because software is essentially changeable.
Although there may be room for statistically informed prediction . . .