Robots Meta Tags
I was just poking around at the source code of one of my product listings and I see that there is no meta information for robots with follow instructions.
Why is that? |
Re: Robots Meta Tags
bump...no one answered this.
Nonetheless, I have another question along the same lines. I downloaded my robots.txt file and noticed that it says User-agent: * # Sitemap example # Sitemap: http://example.com/sitemap.xml and not the actual URL to our sitemap xml file. Why is this? Our site has been up for almost a year and who knows what kind of indexing we have been missing out on. How do I solve this? Should I change the URL in the robots file to https://razorblade.pro/?target=sitemap or should I use https://razorblade.pro/sitemap.xml ? |
Re: Robots Meta Tags
It is not required to have the Sitemap XML link in your robots.txt
There are 3 methods of getting search engines aware of your sitemap XML files: https://www.sitemaps.org/protocol.html#informing The recommended way is submitting the link manually. Please check this article: http://kb.x-cart.com/en/seo_and_promotion/submitting_your_shop_to_search_engines.html Also, on the Sitemap XML configuration page there are two checkboxes that will make XC5 inform Google and Bing servers about new sitemaps when you generate them (this is the third method from the link to sitemaps.org site above). You may also edit your robots.txt and add the link manually. |
Re: Robots Meta Tags
Quote:
Quote:
I'm sure I could submit a bug report or something along those lines about it and then be asked to pay a $99 fee to find out why... Quote:
:x |
Re: Robots Meta Tags
Quote:
Did you submit your sitemap xml manually via Google Webmast Tools? I believe their site immediately informs whether the submitted sitemap is accepted, or not. |
Re: Robots Meta Tags
Quote:
Google renamed the service to "Search Console"... But yes it will say under Search Console how many URLs you submitted and how many URL's google indexed from that submission (doesn't include other pages they may have indexed via deep link crawling or other methods).. For example here is what mine says Sitemaps 655 URLs submitted 221 URLs indexed Since I may have the same url in 2 different site maps, that is why the submitted and indexed are different. |
Re: Robots Meta Tags
I also checked code today and there is no meta robots tag on any pages?
Example: <meta name="robots" content="noindex" /> Why? Can we add this and if so where? |
Re: Robots Meta Tags
The meta tag can easily be added with a custom module. Or you can modify one of the twig files that adds another meta tag and add your to it (you need to do this thought the template edit feature for the changes to stick). Or you can use this module https://market.x-cart.com/addons/custom-http-headers-by-cfl-systems.html to add x-robots-tag header tag - more info https://developers.google.com/search/docs/advanced/robots/robots_meta_tag
robots.txt file is optional and you need to add any entries in there manually. Keep in mind though most bots ignore this file especially the malware bots. |
All times are GMT -8. The time now is 10:26 PM. |
Powered by vBulletin Version 3.5.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.