Magento Exporter Troubleshooting

Problem 1: Inconsistent Product Counts or Insufficient Products Exported

What you see

The number of products exported varies significantly each time the exporter runs, even though you made no changes to your product catalog. Also, all the products are not exported.

Expected

The number of products exported should be consistent every time the exporter runs, as long as no products have been added/removed/modified. Also, all the products should be exported.

Solution

This usually happens if an execution timeout is implemented in the web site or PHP configuration. You can confirm this by noticing that the download time is almost identical every time, and that the last line of the export file is incomplete. The download time will typically be a round number around 1 to 2 minutes, typically 60 seconds or 100 seconds.

The exporter script overrides PHP's max_execution_time setting to 4 hours, but there are other timeouts that could affect how long the script is allowed to run. Check the configurations of PHP, the web server (e.g. Apache), and any web server protection software (e.g. Suhosin).

Problem 2: 504 Gateway Timeouts or 503 Backend fetch failed

What you see

If you have a cache server between your Magento server and the Internet, you may see 504 or 503 error codes in the response when you run the export. This happens because the cache server expects the Magento server to respond within a certain amount of time, typically 30 seconds. But the exporter script may take many minutes to respond and complete, so the cache server returns a "504 Gateway Timeout" or "503 Backend fetch failed" error when its internal request to Magento times out.

Solution

To resolve the problem, turn off caching on the exporter URL. Examples of caching server settings that can cause a problem are listed below.

• Cloudflare Proxy Cache - Cloudflare configure a Page Rule to disable caching/security for a specific path or URL
• Varnish Server Nginx Reverse Proxy Timeout
• Varnish cache server - Bypassing the cache for specific URL patterns

Resolving 403 Forbidden due to CloudFlare

Reproduce Instructions

1. From a shell prompt, run curl --verbose 'https://www.mystore.com/aten_exporter_for_magento.php?Command=Export&Store=1&Password=xxx'
2. The return value contains:

   HTTP/1.1 403 Forbidden
   ...
   cf-mitigated: challenge
   ...
   <title>Just a moment...<title>

3. Go to CloudFlare Dashboard > Security > Events for your store domain
4. Scroll down to Sampled logs
5. You should see a Managed Challenge coming from your IP address, with the same date/time as the curl request, and with a Path of aten_exporter_for_magento.php

Solution

1. Go to CloudFlare Dashboard > Security > WAF for your store domain
2. Click on the Tools tab
3. Under IP Access Rules:
   Enter your IP address
   Select Allow as the Action
   Select All websites in account
   Enter Aten Software Product Exporter in Notes
4. Click the Add button
5. Repeat the test above
6. You will see:

   HTTP/1.1 200 OK
   ...
   x-atensoftware-version: 2021-06-23
   ...
   "sku","name"...
   

Note, adding a Page Rule or WAF Managed rule exception on the URL or specific path did not work for this specific scenario.

JetRails 404 Not Found Error

Problem

You are using Jetrails, and the page gives a 404 error: Page not found. error.

Solution

You may need to "ask Jetrails to open the access for external usage" and "add the provided PHP file to the allowed PHP entry point list."