Magento Exporter Troubleshooting
Problem 1: Inconsistent Product Counts or Insufficient Products Exported
What you see
The number of products exported varies significantly each time the exporter runs, even though you made no changes to your product catalog. Also, all the products are not exported.
Expected
The number of products exported should be consistent every time the exporter runs, as long as no products have been added/removed/modified. Also, all the products should be exported.
Solution
This usually happens if an execution timeout is implemented in the web site or PHP configuration. You can confirm this by noticing that the download time is almost identical every time, and that the last line of the export file is incomplete. The download time will typically be a round number around 1 to 2 minutes, typically 60 seconds or 100 seconds.
The exporter script overrides PHP's max_execution_time setting to 4 hours, but there are other timeouts that could affect how long the script is allowed to run. Check the configurations of PHP, the web server (e.g. Apache), and any web server protection software (e.g. Suhosin).
Problem 2: 504 Gateway Timeouts or 503 Backend fetch failed
What you see
If you have a cache server between your Magento server and the Internet, you may see 504 or 503 error codes in the response when you run the export. This happens because the cache server expects the Magento server to respond within a certain amount of time, typically 30 seconds. But the exporter script may take many minutes to respond and complete, so the cache server returns a "504 Gateway Timeout" or "503 Backend fetch failed" error when its internal request to Magento times out.
Solution
To resolve the problem, turn off caching on the exporter URL. Examples of caching server settings that can cause a problem are listed below.
- Cloudflare Proxy Cache - Cloudflare configure a Page Rule to disable caching/security for a specific path or URL
- Varnish Server Nginx Reverse Proxy Timeout
- Varnish cache server - Bypassing the cache for specific URL patterns