Beauty is not in the eye of the beholder, but the person that encounters beauty always brings something to the table. When you read a great book or listen to great music you have to bring a little something yourself and meet the artist or the piece somewhere in the middle. In fact it's not even a question of whether you should or you should not, it's just unavoidable. I think this inevitability creates the "beauty is in the eye of the beholder" effect.
I might posit a ratio of the artist's mastery to how much the audience must bring to the table. If the artist is a master, the audience hardly has to bring anything of themselves or expend much effort and vice versa. A good rule of thumb is: are you aware that you're listening to music? or did you completely "lose yourself?"
This leads me to refinement. I believe that over-refinement is not beautiful. Refinement might not be the perfect word, but what I'm saying is, a masterful artist doesn't say too little, but he also doesn't say too much. You can imagine how stupid a book would be that describes every last detail to the nth degree. I believe a masterful artist anticipates an audience and almost creates "voids" in the work, where the audience is able to fill in the blanks themselves. Like when I compare the Sagas to The Lord of the Rings, the landscapes in the Sagas are described minimally, whereas the landscapes in LoTR, they are described in detail. For my money, the Sagas creates an atmosphere where I can lose myself, whereas long descriptions sometimes has the odd effect of reminding me that I'm merely reading a book. I hope this makes sense.
I think we should distinguish between art for art's sake, art for beauty's sake, and art for some ideology's sake. I think maybe some conflate art for art's sake with art for beauty's sake, and thus, they are left with an "art for ideology's sake." Art should be for beauty's sake and not for either art's sake or ideology's sake.
Back to "beauty is in the eye of the beholder." The potential problem you run up against is that when you rightfully conclude that beauty is NOT in the eye of the beholder, then you have a tendency to say that beauty is merely a collection of symmetries and ratios, essentially. But the thing about that is, symmetries and ratios are not beauty either, they are simply another way to describe beauty. As others have said, beauty is how all these ratios and symmetries relate to one another all at once and work in unison over time. And, btw, what would the musical analogs be for symmetry or golden ratios? Because music has the added dimension of time, so is there such a thing as temporal symmetry?