You don't have to have alt tag text - it can be empty. But you have to have the tag itself
<img src="xxx" width="xxx" height="xxx" alt="" /> - is correct
<img src="xxx" width="xxx" height="xxx" alt="xxx" /> - is correct
<img src="xxx" width="xxx" height="xxx" /> - is incorrect
The images references (like products, categories, etc) are generated by calling product_thumbnail.tpl and passing values to it for the different tags. If you pass an empty value for a tag (or no value at all) it will not be included. You have to look at this file and make sure alt tag is always displayed no matter what and then look at the template showing "main" image and see if it passes value to alt tag if you want to have one.