I'm not quite sure why ad hominems are seen as a sign of bad reasoning -- some esteemed philosophers used this tactic to great effect (Schopenhauer, Nietzsche), and I'd suggest that some of Nietzsche's most insightful critiques are essentially protracted ad hominem arguments. See his analysis of slave morality and resentiment in "On the Genealogy of Morals" or his criticisms of Wagner -- these passages do more than debunk abstract ideas: they attack the very character of Christians and Wagner. As anyone who thinks long about the matter will know, it is impossible to separate ideas from the people who hold or create them. The idea that philosophers, and people in general, are searching, unbiased, for some abstract, impersonal truth is false, see the first section of Beyond Good and Evil. Thus, the Christian, or Wagner, has not arrived at their beliefs through impartial reasoning -- they have arrived at them since their character is intertwined with these beliefs. This does not only apply to people whose thoughts we hold in contempt -- people like Nietzsche arrived at their conclusions largely based on the quality of their character. Rather than say "remove everything personal from debate," you might as well say "emasculate yourself" -- great debating will show the best of one's qualities.
That's not to say ad hominem cannot be a obstructive force -- most of the debates here and elsewhere on the internet consist of a few central points and then an endlessly repeated cycle of "that was an ad hominem attack" and "why should I debate someone who resorts to personal insults" (both of them implicit ad hominems). But I think it is important to realize that this "fallacy" actually has an important place in discussion and is not equivalent to a big "INVALID" sign.
I don't know if you realize this, but part of your defense for the use of Ad Hominem is an Argument from Authority, another fallacy. And the other part isn't even a logical argument it's just an observation that people do it alot.
Sure there will be a noticeable relationship between ideas and the people who propose them but that relationship isn't necessarily valid or invalid, so the relationship is meaningless.
"YOU made an Ad Hominem attack" is not ad hominem, it is referring to the argument.
"why should I debate someone who uses ad hominems?" is not ad hominem because it is not a counter-argument, it is a refusal to participate, the debate goes unresolved. In the worst case this is a filibuster.
Now things like questioning someone's motives or status (you're just saying that because you're a loser) or accusing them of being a hypocrite I think are particularly obstructive because such responses come very naturally and many observers don't pick up on them.
I think what is important is that we not just avoid committing these errors in our own arguments but become attuned to picking them up in arguments proposed by others.