I tested 4.0.0 and the buttons were quite buggy. A lot of the javascript was broken.
It seems this has changed in 4.0.1. To change them to an image from the ugly "text and go" button.
Code:
{include file="buttons/update.tpl" style="button" href="javascript: document.cartform.submit()" js_to_href="Y"}
The
style="button" is what you are looking for to make it an image.
I hope this helps.