How to Resolve Magento Re- Indexing Issues

If you’re a Magento develop or a Magento store owner than you must have faced this issue at least once. The re-indexing of your item catalogue or pages within a catalogue is a common and a pricky issue with both Magento 2.0 and the latest Magento 2.1.3. And rectifying this error has been a mixed reaction, most tricks work and some don’t.

As an ecommerce development company working and developing multiple projects in Magento Development Services since last 8 years, we’ve faced many issues both within our site and client’s Magento Ecommerce stores and we know what to do with them.

Why do you face re-indexing problems with Magento?
For faster processing and rapid record retrieval Magento uses database indexes of products. Ideally, as you update your products or portfolio, it should trigger re-indexing of records. But, sometimes this does not happen with Magento for variety of reasons and you see various sitemap and indexing errors.

If you’re looking for a solution or an answer to a nagging issue you’re facing with Magento store sitemap index, then we’ve listed out few of them here which should get you on the right path to solving them. Here is the list:

Q. My Sitemap XML does not convert to HTTPS
A. It’s because you have not set your secure base URL as HTTPS. Please go to Admin panel, Select URLs on Storefront = Yes, then select Secure URLs in Admin = Yes and change setting to Secure Base URL = starts with HTTPS. And you will then start seeing your Sitemap XML URLS will convert to HTTPS in few hours.

Q. Can I transfer my sitemap from Magento 1.0 to Magento 2.0 or Magento 2.1.3?
A. Yes, you can do it technically. However, you will need to take support of some custom Magento Extensions. Create a new FO controller and manage template display/layout.

You can also create a new module with the Magento 2.0 or Magento 2.1.3 and run XML file there with all your URLs. Generate a static file in HTML which can be taken up by a Web server to a custom URL.

Q. I can’t see my sitemap in Magento 2.0.
A. By default, Magento does not provide with an HTML sitemap. You will need to either create one from the scratch or use an extension for it.

Q. I cannot generate a sitemap using HTTPS.
A. If you’re using an SSL certified site and facing this issue, then simply remove the .htaccess lines with extra code and enter https:// address under your System – Configuration – Web – Unsecure Base URL as well as Secure Base URL. You can also try using /media/ as a path for sitempa.xml to access the site permissions, in case you need it.

Q. Most common Re-indexer error. I’m getting a “Cannot initialize the indexer process” message.
A. It is one of the most common Magento error message with a large store with multiple product stores within a site or a web of websites. So, it has something to with your PHP server’s memory and CLI indexing. You can follow these steps to debug it permanently:

Run Indexing from the CLI – Go to CLI and at the root of your Magento installation and change attributes to: php shell/indexer.php –reindex catalog_product_price;

Var folders- If you do not have any folder in var/ then, Make sure that var/ folder has the correct file permissions for the admin ownership as Magento needs these folders for re-indexing. You can go to var/ folder in ‘tmp/magento/var/” If your var folder is located there, it is because you have a permissions issue.

Changing your memory limit- Go to .htaceess within your site’s root_ini and select root ini_set (‘memory_limit’,’2048M’) and your issue for the memory should get resolved.

Q. I can’t see some active categories in sitemap.xml in Magento
A. This happens when you are using active form with limitations. Usually, active categories are fetched from backend and shown in frontend, but sometimes due to Meta field limitations, you are unable to see them on sitemap.xml. We suggest that you add values in the Meta fields and remove limitations from Active forms to rectify this error.
Q. I want to remove a specific URL from my Sitemap.xml in Magento
A. To do this, you will need to create a separate module, for example Yourname_sitemap with a proper rewrite model like Manage_sitemap_model_sitemap.

Now, go to app/code [local/community]/YourNameSpace/Sitemap/Model and create a class sitemap.php which will extend to the original sitemap model class. You will be able to copy and adjust the generateXML () method with it and keep the rest in parent class.
Add the URL in the new class that you have created and that will not show in the parent sitemap.xml file.

Q. My robots.txt automatically adds Magento/sitemap.xml
A. Sometimes, even when you try to remove Magento/sitemap.txt file from stores/general/design it automatically gets added to robots.txt file.

And if you try to forcefully remove it, the sitemap indexing will give you a 404 error.

The best way is to enable submission of Robots.txt to no while you’re publishing changes for that period and set file limits for your sitemap as shown in the image.

Q. Don’t want to include CMS page in my sitemap.xml
A. In a large Magento Ecommerce store, there are hundreds of CMS pages and Magento allows creation of sitemap.xml via Magento Admin Panel. But, when you create a sitemap from there, by default all the CMS pages will get included.

To fix this, you will need to create an installer which can create new fileds (is_used_for_sitemap) at cms_page table in database. Then view it from Admin cms page view, it should be in a dropdown and rewrite a resource model class Manage_Sitemap_Model_Resource_CMS_page and
[ModuleNameSpace]_[ModuleName]_Model_Resource_Cms_Pag
and – that function getCollection () add this field as condition (is_used_for_sitemap=1).
This should resolve the issue once and for all.

There are potentially hundreds of unexplored issues and fixes that require easy fixes. We’ve noticed that with Magento 2.0 there have been very few bugs or issues like these reported so far. We regularly post about Magento platform related updates and our dedicated full-stack Magento Developers can resolve any issues with the Magento platform.

If you’re facing any issues with your Magento webstore then, just shout out to our great Magento professionals on hello@i-verve.com and we will surely get back to you at the earliest with a quick and long term fix.