Click here to close this tab and return to the app.

Support  →  Topics  → 

Magento Product Data Feeds Setup Guide

Magento Extension Installation

We recommend installing our exporter script, but you can also install our Magento Extension via Magento Connect.

OPTION 1 (PREFERRED) - Script Installation

Upload our exporter script to your website using the Magento Product Export Script Setup Instructions.

OPTION 2 - Automated Installation (via Magento Connect)

  1. Log into your Magento Admin Panel
  2. Go to System > Magento Connect > Magento Connect Manager.
  3. Log in again to MagentoConnect Manager
  4. If you see a message saying Warning: Your Magento folder does not have sufficient write permissions., please configure the FTP Connection from the MagentoConnect Manager Settings page.
  5. Next, go to Aten Software Product Export for Data Feeds and click Install Now to obtain the Extension Key. (You can choose either Magento Connect 1.0 or Magento Connect 2.0 format.)
  6. Go back to the MagentoConnect Manager page, and enter the extension key into the box labeled: Paste extension key to install:
  7. Click Install
  8. Click Return to Admin
  9. Go to System > Configuration in the main menu
  10. Select Advanced > Advanced in the left side-bar
  11. Expand the Disable Modules Output section. If you see AtenSoftware_ProductExport with a setting of Enable, then the extension has been correctly installed.
  12. Continue with Add Magento API User and Role.

OPTION 3 - Manual Extension Installation

  1. Visit Aten Software Product Export for Data Feeds, and install the extension into your Magento Admin.
    1. Use these steps for a manual installation:
    2. Download the AtenSoftware_ProductExport-14.3.13.0.tgz package.
    3. Unzip it to a temporary folder
    4. Copy the AtenSoftware folder into ./app/code/community/
    5. Copy the modules/AtenSoftware_ProductExport.xml file into ./app/etc/ folder so its final path is: ./app/etc/modules/ AtenSoftware_ProductExport.xml
  2. The extension key is:   AtenSoftware_ProductExport
  3. To verify that the package is installed correctly, go to System > Configuration > Advanced and expand the Disable Modules Output section. If you see AtenSoftware_ProductExport with a setting of Enable, then the extension has been correctly installed.
Add Magento API User and Role for Magento Connect Extension

These instructions apply to the Magento Connect extension.

  1. Go to System > Web Services > (SOAP/XML-RPC) Roles in your Magento Admin.
  2. Click Add New Role
  3. Enter atensoftware as the Role Name
  4. Click the Role Resources tab
  5. Check the Aten Software > Get Product Export checkbox
  6. Check the Core checkbox
  7. Click the Save Role button
  8. Go to System > Web Services > (SOAP/XML-RPC) Users in your Magento Admin.
  9. Click Add New User
  10. Enter the following:
    User Name: atensoftware
    First Name: Aten
    Last Name: Software
    Email: support email address
    API Key: Make up a a password here. Here is a random one for your convenience: fnv24y97
    API Key Confirmation: Re-enter the password
    This account is: Active
  11. Click the User Role tab
  12. Click the radio button for the atensoftware role
  13. Click Save User
  14. Continue with Configure Feed Source Settings for Magento Connect Extension.
Configure Feed Source Settings for Magento Connect Extension
  1. Go to System > Manage Stores in your Magento Admin.
  2. Hover your mouse over the Store View Name of the store you want to use, and note the ID number. It is typically 1, but could be anywhere from 1 to 10 or more. If you do not see it as a tooltip, look at the URL of the link to find the store_id parameter.
  3. In our site, go to Manage Feed > Modify Settings for your Magento feed (assuming you have added it already).
  4. Enter the API URL as follows into the Product Catalog URL setting:
    http://atensoftware:password
    @www.yourdomain.com
    /api/?wsdl&storeId=1
    Make sure to replace the bold parts in the above URL with the API username, API password, domain, and ID number of your store.
  5. To determine the correct API URL for your store site, you can look at the Base URL setting under System > Configuration in the main menu, in the General > Web > Unsecure section. The URL will be that with /api at the end.
  6. To test the API connection, go to Manage Feed > Test Settings after saving the settings. You will see the results of a test connection with detailed error message.

Technical Settings

The settings below are technical in nature and are generally not required. Please contact us if you see internal system errors or products are missing, and we can assist you with the settings.

includeDisabled
Default value is "0"
Set to "1" to include disabled items in the export, "0" otherwise.
Set this if you are missing a lot of products.
itemsPerRequest
Default value is 5000
Number of items to fetch per request.
Set lower to lower server memory requirements.
excludeOutOfStock
Set to "1" to exclude out of stock items in the export, "0" otherwise.
Default value is "0"
Use this setting if you have a large number of out of stock items in your export that is slowing the export down.

Add the settings at the end of the URL in the following format. The setting names are case-sensitive:

&includeDisabled=1& itemsPerRequest=100& excludeOutOfStock=1

If you leave out any setting, the default value will be used for that setting.

Note: There is a known bug where if the itemsPerRequest is set too low, some items may fail to export. If that happens, set includeDisabled to "1" to ensure all the products are exported. The disabled items can be filtered out of the feed using product filters instead.

Resolving the error: SOAP extension is not loaded

You may receive the error "SOAP extension is not loaded" when setting up a Magento Connect data feed.

Our Magento Connect extension connects to your Magento Store API via SOAP requests, and it requires the PHP SOAP extension to be installed on your website.

You can verify if PHP SOAP is installed on your website by accessing this page:
http://www.mystore.com/shop/api/v2_soap
(Remember to change www.mystore.com/shop to your Magento shop URL.)

Usually, you can simply ask the web hosting company that hosts your website to install PHP SOAP for you. Note that it can not be installed via the Magento admin login. It has to be installed by the web hosting company, or by someone with root access to the web server.

Please see the Api Error - 0 SOAP extension is not loaded for other potential fixes.

Magento Visibility and Status Constants

In the Magento catalog, the 'visibility' field could have the following possible values:

VISIBILITY_BOTH = 4
VISIBILITY_IN_CATALOG = 2
VISIBILITY_IN_SEARCH = 3
VISIBILITY_NOT_VISIBLE = 1
Mage_Catalog_Model_Product_Visibility

For Shopping Feeds, it is generally advisable to exclude items where 'visibility' != 4.

If you have are using the Magento API-based export method, then visibility values will be 'Catalog, Search', 'Catalog', 'Search', and 'Not Visible Individually'. In that case, the filter should be set up as 'visibility' != 'Not Visible Individually'.

In the Magento product catalog, the 'status' field could have the following possible values:

STATUS_DISABLED = 2
STATUS_ENABLED = 1
Mage_Catalog_Model_Product_Status

By default, our exporter extension excludes items where 'status' = 2, so no special filters are typically required.

Export All Products via Magento API

Instead of using our exporter script or extension, our system can connect directly to the Magento API to export product data. We recommend using this method only if there are technical issues with the exporter script/extension, because exporting via the API is very slow.

Examples of When to Use API-based Export

  • url_key or url_path not getting exported
  • category information not exported or missing
  • category/product flat tables are disabled in your Magento settings
  • other problems getting the extension or exporter script to work

Set up Magento API Role and User for Product Export

  1. Go to System > Configuration > Services > Magento Core API in your Magento Admin.
  2. Set the General Settings > Enable WSDL Cache setting to Yes, and click the Save Config button.
  3. Go to System > Web Services > (SOAP/XML-RPC) Roles in your Magento Admin.
  4. Click Add New Role
  5. Enter atensoftware as the Role Name
  6. Click the Role Resources tab
  7. Check the Core checkbox
  8. Check the Catalog checkbox
  9. Check the Catalog Inventory checkbox
  10. (Our system only needs read access. For additional security, you can uncheck all the write permissions under the Catalog and Catalog Inventory trees. These include the following: Update, Assign, Delete, Remove, Add, Create, Rename, and Move)
  11. Click the Save Role button
  12. Go to System > Web Services > (SOAP/XML-RPC) Users in your Magento Admin.
  13. Click Add New User
  14. Enter the following:
    User Name: atensoftware
    First Name: Aten
    Last Name: Software
    Email: support email address
    API Key: Make up a a password here. Here is a random one for your convenience: ynk4h7e1
    API Key Confirmation: Re-enter the password
    This account is: Active
  15. Click the User Role tab
  16. Click the radio button for the atensoftware role
  17. Click Save User

Configure your Feed to Export Products via the Magento API

Go to Manage Feed > Modify Settings, and set the Product Catalog URL to the following:

http://atensoftware:password@ www.yourdomain.com/api/?wsdl

Enable Flat Product and Category Tables in Magento

To enable flat table support in Magento, go to System > Configuration > Catalog > Frontend in your administration dashboard.

Change the Use Flat Catalog Category and Use Flat Catalog Product settings to Yes and click the Save button.

In Magento 2+, edit vendor\magento\module-catalog\etc\config.xml to set flat_catalog_category system value.

After enabling the settings, go to System > Index Management and update the flat data indexes.

Magento Exporter Troubleshooting

Problem 1: Inconsistent Product Counts or Insufficient Products Exported

What you see

The number of products exported varies significantly each time the exporter runs, even though you made no changes to your product catalog. Also, all the products are not exported.

Expected

The number of products exported should be consistent every time the exporter runs, as long as no products have been added/removed/modified. Also, all the products should be exported.

Solution

This usually happens if an execution timeout is implemented in the web site or PHP configuration. You can confirm this by noticing that the download time is almost identical every time, and that the last line of the export file is incomplete. The download time will typically be a round number around 1 to 2 minutes, typically 60 seconds or 100 seconds.

The exporter script overrides PHP's max_execution_time setting to 4 hours, but there are other timeouts that could affect how long the script is allowed to run. Check the configurations of PHP, the web server (e.g. Apache), and any web server protection software (e.g. Suhosin).

Problem 2: 504 Gateway Timeouts or 503 Backend fetch failed

What you see

If you have a cache server between your Magento server and the Internet, you may see 504 or 503 error codes in the response when you run the export. This happens because the cache server expects the Magento server to respond within a certain amount of time, typically 30 seconds. But the exporter script may take many minutes to respond and complete, so the cache server returns a "504 Gateway Timeout" or "503 Backend fetch failed" error when its internal request to Magento times out.

Solution

To resolve the problem, turn off caching on the exporter URL. Examples of caching server settings that can cause a problem are listed below.

Resolving 403 Forbidden due to CloudFlare

Reproduce Instructions

  1. From a shell prompt, run curl --verbose 'https://www.mystore.com/aten_exporter_for_magento.php?Command=Export&Store=1&Password=xxx'
  2. The return value contains: 
    HTTP/1.1 403 Forbidden
...
cf-mitigated: challenge
...
<title>Just a moment...<title>
  3. Go to CloudFlare Dashboard > Security > Events for your store domain
  4. Scroll down to Sampled logs
  5. You should see a Managed Challenge coming from your IP address, with the same date/time as the curl request, and with a Path of aten_exporter_for_magento.php

Solution

  1. Go to CloudFlare Dashboard > Security > WAF for your store domain
  2. Click on the Tools tab
  3. Under IP Access Rules:
    Enter your IP address
    Select Allow as the Action
    Select All websites in account
    Enter Aten Software Product Exporter in Notes
  4. Click the Add button
  5. Repeat the test above
  6. You will see: 
    HTTP/1.1 200 OK
...
x-atensoftware-version: 2021-06-23
...
"sku","name"...

Note, adding a Page Rule or WAF Managed rule exception on the URL or specific path did not work for this specific scenario.

Feedback