I haven't done this myself, but this info was in a post by Stephen Hatton:
To have your icon saved with the links, you need to put the following line in the home.tpl file in your Б─°/myshop/schemes/templates/2-columns/customer/home.tplБ─² file (or 2-columns_reversed, 3-columns_reversed, small_shop directory):
Б─°<LINK REL="SHORTCUT ICON" HREF="/favicon.ico">Б─² without the outside quotes just before the Б─°</HEAD>Б─² line.
This will embed the icon link into the header of each page.
Maybe it will help you.
Paul Dodman
e-business consultant
Lumino Internet
luminointernet.co.uk