I did this by simply changing the code in skin1/buttons/add_to_cart.tpl to:
[img]{$ImagesDir}/buttons/add_to_cart.gif[/img]
Make sure your image path is correct and it should work fine providing the original reference to the button template is correct:
{include file="buttons/add_to_cart.tpl"}
Hope it's the same in version 4! My version is 3.4.8 though my sig may say different.
HTH
Mat
PS. This works with all buttons, but will not let you use mutiple languages, obviously.