Follow us on Twitter X-Cart on Facebook Wiki
Shopping cart software Solutions for online shops and malls

Robots Meta Tags

 
Reply
   X-Cart forums > X-Cart 5 > General questions (X-Cart 5)
 
Thread Tools Search this Thread
  #1  
Old 09-22-2016, 01:22 PM
 
BrandonLR BrandonLR is offline
 

Senior Member
  
Join Date: Jan 2009
Posts: 161
 

Default Robots Meta Tags

I was just poking around at the source code of one of my product listings and I see that there is no meta information for robots with follow instructions.
Why is that?
__________________
X-Cart Business 5.4.1.29
Reply With Quote
  #2  
Old 01-05-2017, 01:35 PM
 
BrandonLR BrandonLR is offline
 

Senior Member
  
Join Date: Jan 2009
Posts: 161
 

Default Re: Robots Meta Tags

bump...no one answered this.

Nonetheless, I have another question along the same lines.

I downloaded my robots.txt file and noticed that it says
User-agent: *
# Sitemap example
# Sitemap: http://example.com/sitemap.xml

and not the actual URL to our sitemap xml file.
Why is this? Our site has been up for almost a year and who knows what kind of indexing we have been missing out on.

How do I solve this? Should I change the URL in the robots file to https://razorblade.pro/?target=sitemap or should I use https://razorblade.pro/sitemap.xml ?
__________________
X-Cart Business 5.4.1.29
Reply With Quote
  #3  
Old 01-05-2017, 11:52 PM
  qualiteam's Avatar 
qualiteam qualiteam is offline
 

X-Guru
  
Join Date: Dec 2010
Posts: 6,373
 

Default Re: Robots Meta Tags

It is not required to have the Sitemap XML link in your robots.txt

There are 3 methods of getting search engines aware of your sitemap XML files:
https://www.sitemaps.org/protocol.html#informing

The recommended way is submitting the link manually. Please check this article:
http://kb.x-cart.com/en/seo_and_promotion/submitting_your_shop_to_search_engines.html

Also, on the Sitemap XML configuration page there are two checkboxes that will make XC5 inform Google and Bing servers about new sitemaps when you generate them (this is the third method from the link to sitemaps.org site above).

You may also edit your robots.txt and add the link manually.
__________________
Alex Solovev,
Qualiteam

---

User manual Video tutorials X-Cart FAQ

You are welcome to press "Thanks" button
if you find this post useful

Click here to learn how to apply patches

X-Cart Extensions
Reply With Quote
  #4  
Old 01-06-2017, 05:44 AM
 
BrandonLR BrandonLR is offline
 

Senior Member
  
Join Date: Jan 2009
Posts: 161
 

Default Re: Robots Meta Tags

Quote:
Originally Posted by qualiteam
It is not required to have the Sitemap XML link in your robots.txt
Apparently it is because Google, to name just one, had no idea of my sitemap.


Quote:
Originally Posted by qualiteam
The recommended way is submitting the link manually. Please check this article:
http://kb.x-cart.com/en/seo_and_promotion/submitting_your_shop_to_search_engines.html

Also, on the Sitemap XML configuration page there are two checkboxes that will make XC5 inform Google and Bing servers about new sitemaps when you generate them (this is the third method from the link to sitemaps.org site above).
Did this on multiple occassions over the last 11 months and yet, again, Google had no record of my sitemap and my robots.txt file still said "example.com"
I'm sure I could submit a bug report or something along those lines about it and then be asked to pay a $99 fee to find out why...

Quote:
Originally Posted by qualiteam
You may also edit your robots.txt and add the link manually.
That's what I did and everything seems to be working fine now.
__________________
X-Cart Business 5.4.1.29
Reply With Quote
  #5  
Old 01-08-2017, 09:46 PM
  qualiteam's Avatar 
qualiteam qualiteam is offline
 

X-Guru
  
Join Date: Dec 2010
Posts: 6,373
 

Default Re: Robots Meta Tags

Quote:
Originally Posted by BrandonLR
Did this on multiple occassions over the last 11 months and yet, again, Google had no record of my sitemap and my robots.txt file still said "example.com"

Did you submit your sitemap xml manually via Google Webmast Tools? I believe their site immediately informs whether the submitted sitemap is accepted, or not.
__________________
Alex Solovev,
Qualiteam

---

User manual Video tutorials X-Cart FAQ

You are welcome to press "Thanks" button
if you find this post useful

Click here to learn how to apply patches

X-Cart Extensions
Reply With Quote
  #6  
Old 01-09-2017, 01:05 PM
  GlasPak's Avatar 
GlasPak GlasPak is offline
 

eXpert
  
Join Date: May 2015
Posts: 241
 

Default Re: Robots Meta Tags

Quote:
Originally Posted by qualiteam
Did you submit your sitemap xml manually via Google Webmast Tools? I believe their site immediately informs whether the submitted sitemap is accepted, or not.

Google renamed the service to "Search Console"... But yes it will say under Search Console how many URLs you submitted and how many URL's google indexed from that submission (doesn't include other pages they may have indexed via deep link crawling or other methods)..

For example here is what mine says

Sitemaps
655 URLs submitted
221 URLs indexed

Since I may have the same url in 2 different site maps, that is why the submitted and indexed are different.
__________________
Business License
https://glaspak.com
GoDaddy VPS - Apache with WHM
2GB ECC DDR3 1600MHz
2x Intel E5-2630L v2 Hex Core 2.4/2.8ghz
15 MB proc cache
60 GB 10k RPM hdd
PHP 7.0.15
MariaDB 10.1.21 with InnoDB engine
Reply With Quote
  #7  
Old 12-27-2021, 05:30 AM
 
lostdog9 lostdog9 is offline
 

Member
  
Join Date: May 2014
Location: USA
Posts: 23
 

Default Re: Robots Meta Tags

I also checked code today and there is no meta robots tag on any pages?

Example: <meta name="robots" content="noindex" />

Why?

Can we add this and if so where?
__________________
5.3.5.8
Assigned Products
Backorder / Preorder
Banner System
Bestsellers
Bulk Editing
Catalog
Catalog Subtitles
Category Second Description
Product Feeds
Product Variants
Reply With Quote
  #8  
Old 12-28-2021, 07:21 AM
  cflsystems's Avatar 
cflsystems cflsystems is offline
 

Veteran
  
Join Date: Apr 2007
Posts: 14,189
 

Default Re: Robots Meta Tags

The meta tag can easily be added with a custom module. Or you can modify one of the twig files that adds another meta tag and add your to it (you need to do this thought the template edit feature for the changes to stick). Or you can use this module https://market.x-cart.com/addons/custom-http-headers-by-cfl-systems.html to add x-robots-tag header tag - more info https://developers.google.com/search/docs/advanced/robots/robots_meta_tag


robots.txt file is optional and you need to add any entries in there manually. Keep in mind though most bots ignore this file especially the malware bots.
__________________
Steve Stoyanov
CFLSystems.com
Web Development
Reply With Quote
Reply
   X-Cart forums > X-Cart 5 > General questions (X-Cart 5)


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -8. The time now is 07:49 PM.

   

 
X-Cart forums © 2001-2020