The reason is to ensure that whoever did the work in your basement did it to minimum code to and that it is safe for you and the people who buy your home next. If you don't get a permit your insurance may not cover expenses if you have a flood or fire. I see many basements that do not have proper insulation which can make the basement cold in the winter time. Poorly finished basements can also get mouldy and smell very damp that could cause health issues. I have seen rotten studs when gutting basements because of all the moisture. Many people "cheap out" in basements and they just don't last. If your contractor says you don't need a permit then I would suggest you find a different contractor.
You also need to get an ESA permit for all electrical being done in your basement. The home owner can do the work themselves as long as they get the permit but a licensed electrician must do the work if it is contracted out. Not getting the ESA permit could also cause problems with the insurance company and some building permits require it for final inspections.