Hi anela2003,
What about trying this below. Adapt where you see fit (I've removed a quite a bit of code as we're concentrating on the table itself and made it html valid in the process)...
Code:
{if $subcategories}
{assign var="items_per_row" value=5}
<table class="sublisttable">
<tr class="sublisttr">
{foreach from=$subcategories item=subcat name=loopy}
<td class="subclist">
<a href="home.php?cat={ $subcat.categoryid }"><img src="{if $subcat.icon_url}{$subcat.icon_url}{else}{$xcart_web_dir}/default_image.gif{/if}" alt="Priority Windows Custom Products" width="150" height="150" /><br />
<span class="ItemsList">{ $subcat.category|escape }</span><br /><img src="images/customize.png" width="150" height="25" alt="" /></a>
</td>
{if $smarty.foreach.loopy.iteration mod $items_per_row eq 0}
</tr><tr class="sublisttr">
{/if}
{/foreach}
</tr>
</table>
{/if}
Hope this gets in you in the right direction.