Good points, all.
I guess my observation is this: great albums start with a mental/emotional spark. Some vision, some concept, something toward which one is yearning.
That's the goal. When it gets supplanted by popularity/money and/or exhaustion, it's over for higher quality output.
Still, the ability sleeps within them. More talented people are hard to find.