I recommend Klemperer, Kletzki, Szell, Kubelik, Weingartner, Sotin, Toscanini and Furtwängler, as far as entire symphony cycles go.
Actually, if you can handle another viewpoint, here it is:
Avoid Klemperer like the plague. He turns Beethoven into schmaltz. I spent many years disliking Beethoven because of the passionless, sentimental, weeping NPR-liberal style voice given to it by Klemperer. On a scale of one to ten, he is a 10 technically and a 0 artistically. Avoid like the fucking plague.
I feel similarly about Weingartner. Toscanini, Furtwangler and von Karajan get my vote. I would recommend for a beginner to go with von Karajan because of all of these, he understands the music the best, even if we can quibble over details. But if we look into that quibbling, we'll see that most of the people doing it in public (not in this forum) are talking to hear themselves talk. von Karajan does the best overall job of interpreting the spirit of the music, even if he takes some liberties (tempo) and has a monochromatic voice -- he's not trying to be the artist here, but the conduit of the artist, so that's appropriate. The trendy composers either try to be "100% historically-musically accurate" (a total illusion) or to dress up the music in some unnecessary accessories like hipsters. Don't fall for this bad psychology -- stick to von Karajan and Furtwangler.
As far as buying symphony cycles, I've never found one that is 100% perfect, but that's not what you want. You buy symphony cycles to learn the artist from the consistent viewpoint of a conductor who understands not just the music but the spirit. This is why people buy von Karajan for Bruckner and Beethoven and Jochum for Bruckner. Celibdache is OK by my lights but too modern in his dramatic interpretation. For the best man on Bruckner's 4th, try Salonen.
I've listened to classical music my whole life and am not inexperienced with art, so although I'm not an "authority" on these works, I know of what I speak. I know that some idiot will say "you seem to not like conductors of a certain ethnic background" and I will say this is coincidental, but that for any type of music, the best person to play it is a person who grew up in the culture that fomented it. I have heard enough idiots convert classical music into trash by stripping it of its real emotion and substituting furniture-store background music emotions in the most cheesy, schmaltzy way -- and all the press and the music instructors run their mouths like robots praising them. Don't believe the hype.