It's not likely you are hurting your ends, unless you are using a conditioner that has something in it to which your hair does not react well (many people have problems with silicones).  As long as you are using conditioner that is right for your hair, you're more likely to cause damage by under-conditioning rather than over-conditioning.