Is it harmful not to change brake and transmission fluids at the times recommended.?

My service provider informed me that brake and transmission fluids are dirty and contaminated and recommended changing them--a not inexpensive proposition. How harmful would it be to my car if I never did this? If I must do it how long can I safely delay it? I keep my cars on average 8-10 years before resale.