In 2002 I had my hair permed, in 2003 I had it straightened. 

-I can hear you all screaming and some rolling over in graves-

Although my trimmer-guy says I have no damage I wonder if this will make my hair grow slower or not all.  Do you think this will keep my hair from growing.

Chemical processes affect the hair shaft which is dead.  Your hair growth comes from the hair follicle.  This means that anything you have done to your hair in the past will not affect your hair growth. 

You are very lucky to not have suffered significant damage from the perm and straightening processes.  Perms always fried my hair.