php-fpm, nginx, bad hoodoo magic, and the dreaded 502 gateway error on magento sites

When segmentation faults collide ….

If anything disrupts nginx’s ability to talk to php-fpm, you’ll see the 502 gateway error. Sometimes, that’s because the server guys did something. Sometimes, that’s because the dev guys did something. Sometimes, it’s really, really, really hard to tell.

If your host assures you it’s your code, and they are positive they have not updated anything on the server (php version, nginx version, anything at all), and more importantly, if the 502 gateway is intermittent and always worse under load, you need to consider your code. It may function, but function so poorly it kills the server.

On Magento sites, look for long running queries. You can stack trace by enabling the zend logger in lib/Varien/Db/Adapter/Pdo/Mysql.php.

One of the biggest coding mistakes in Magento customizations is inefficient queries. Running SQL queries inside a loop, for example, can really put a drain on your server. A long running query can consume so much memory it crashes php-fpm which times out the response from nginx and gives you a 502 gateway error, even though your server is running just fine as far as you can see.

Running SQL queries is a very expensive operation, and doing it in a loop tends to make it even worse. Instead of doing that we should use data collections to load models and then process the items in the collection.

Instead of:

foreach ($this->getProductIds() as $productId){
    $product = Mage::getModel('catalog/product')->load($productId);
    $this->processProduct($product);
}

Do this:

$collection = Mage:getResourceModel('catalog/product_collection')
    ->addFieldsToFilter('entity_id', array($this->getProductIds()))
    ->addAttributeToSelect(array('name'));

foreach ($collection as $product){
    $this->processProduct($product);
}

Especially be on the lookout for queries run through an adapter. Method fetchAll() used to fetch and iterate over larger result sets will lead to a very long execution time (again, with the memory thing and the time out thing and the 502 gateway thing). The better solution is to fetch the results row by row using the fetch() method.

Assuming you declared/initialized your adapter already, instead of this:

$rowSet = $adapter->fetchAll($select);
foreach ($rowSet as $row) {
    //process row
}

Do this:

$query = $adapter()->query($select);

while ($row = $query->fetch()) {
    //process row
}

Q: A custom frontend controller will extend which one of the following classes?

All Action Controller need Mage_Core_Controller_Front_Action as an ancestor.

The best example of this in action is a simple HelloWorld module that is configured with routes and has a Create Action Controller for the routes.

Directory Structure:

app/code/local/nodwell/Helloworld/Block
app/code/local/nodwell/Helloworld/controllers
app/code/local/nodwell/Helloworld/etc
app/code/local/nodwell/Helloworld/Helper
app/code/local/nodwell/Helloworld/Model
app/code/local/nodwell/Helloworld/sql

We need a configuration file (PATH: app/code/local/nodwell/Helloworld/etc/config.xml):


<config>    
    <modules>
        <nodwell_Helloworld>
            <version>1.0.0</version>
        </nodwell_Helloworld>
    </modules>
</config>

and a modules file to activate our module (PATH: app/etc/modules/nodwell_Helloworld.xml):


<config>
    <modules>
        <nodwell_Helloworld>
            <active>true</active>
            <codePool>local</codePool>
        </nodwell_Helloworld>
    </modules>
</config>

Now, the module exists, and we can begin to add the code to make it do stuff. We need to configure a route in the config.xml. The route turns a URL into an Action Controller and a method. In our case, it will act on URLs that start with /helloworld, as in http://magento.nodwell.net/helloworld/*. So, add a frontend routers section to the config.xml:


<config>    
    <modules>
        <nodwell_Helloworld>
            <version>1.0.0</version>
        </nodwell_Helloworld>
    </modules>
    <frontend>
        <routers>
            <helloworld>
                <use>standard</use>
                <args>
                    <module>nodwell_Helloworld</module>
                    <frontName>helloworld</frontName>
                </args>
            </helloworld>
        </routers>  
    </frontend>
</config>
 

Now, we create the actual Action Controller (PATH: app/code/local/nodwell/Helloworld/controllers/IndexController.php):

class nodwell_Helloworld_IndexController extends Mage_Core_Controller_Front_Action {        
    public function indexAction() {
        echo 'Hello Index!';
    }
}

Code Generation in Magento 2

When and Why Does Code Get Created in /var/generation?

Overview

Code generation can be triggered in two ways:

1. On the fly – when the system tries to autoload a class, if it doesn’t find it, it generates it. (slower)

delete the MAGENTO_ROOT/var/generation directory

2. Command-line – process goes through the system, inspects the code, and generates the necessary classes it might need. (speedier)

run the magento setup:di:compile command from terminal on web server

Code Generation did not exist in Magento 1. In Magento 2, code generation is deployed to support a number of core concepts.

The system generates several class types – the 3 most important are Factories, Proxies, and Plugins.

Factories

Factories are used to instantiate objects that cannot be injected automatically, objects that it doesn’t make sense to create with a generic mechanism, such as those which contain data from the database.

  1. Developer declares a dependency on factory in constructor ($productFactory)
  2. Object manager injects this dependency
  3. Developer can access create() method (the only method in factory object) to create as many Product instances as he wants.

The purpose of factories is to delegate object instantiation to object managers.

Proxies

Magento 2 uses the object manager to create all the dependencies and allows only one type of dependency injection: constructor injection. Thus, you cannot instantiate an object without passing all the dependencies as they are required. Proxies allow you to pass in optional dependencies.

Developer never touches PHP files to use proxy, only inside the di.xml

Whenever any proxy method is called for the first time, the original instance gets created with all its dependencies. The purpose of a proxy is to delay creation of an instance (and its dependencies) until the very first usage.

Plugins (Interceptors)

Plugins are the primary customization mechanisms for Magento 2 which replace class rewrites. Plugins allow you to hook in and do something before, after or around any public method of the application.

  1. Developer writes a plugin class depending on the requirements
  2. Developer registers a plugin in di.xml

The system generates the interceptor class. The generator tool will generate only the methods that you rewrite.

Installing Magento 2 with Composer

If you’re looking for something amusing to do some rainy afternoon, don’t install Magento 2. 😉

Before You Start

Before you start your installation, make sure your web server meets the minimum system requirements:

  • A Linux x86-64 operating system
  • 2GB of RAM
  • Composer (latest stable version)
  • Apache 2.2 or 2.4 with mod_rewrite enabled or nginx 1.8 or latest mainline version
  • MySQL 5.6 (MariaDB and Percona are compatible)
  • PHP 5.6x, 5.5.22 or higher, 7.02 up to 7.1.0 but not 7.0.5 – required PHP extensions:
    • bc-math
    • curl
    • gd, ImageMagick 6.3.7 or both
    • intl
    • mbstring
    • mcrypt
    • mhash
    • openssl
    • PDO/MySQL
    • SimpleXML
    • soap
    • xml
    • xsl
    • zip

For more information, see the magento dev docs.

Magento Access Keys

My Account
My Account Marketplace Tab – Click on My Access Keys link

Next, you need to log in to your account on the Magento Marketplace and request a key pair to authenticate the install with. It’s free. Sign up.

Your Magento Access Keys Screen
Your Magento Access Keys Screen

On the next screen, click the button to Create a New Access Key and give it any name you like. Leave this window open because you are going to need the Public Key and Private Key that it generates when you create a new access key.

For more information, see the magento dev docs.

Install Via Composer

In a terminal, on the command line, in the root of your magento 2 directory, enter the following:

composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition

It is going to prompt you for a username. Enter the Public Key you got from the Magento Marketplace. It will prompt you for a password. Enter the Private Key you got from the Magento Marketplace.

Install Via Composer
Install Via Composer

Install Sample Data

Enter the following two commands in the terminal from your magento2 root directory:

php bin/magento sampledata:deploy
php bin/magento setup:upgrade
Errors!
Errors!

If you are really, really inordinately lucky, you now have a working magento 2 site with sample data installed. Most likely, though, you got some errors when you tried to install the sample data, and you’re stuck.

If you try to install sample data, and get a list of packages that “could not be found in any version,” don’t panic, there’s a GIT work around.

Install Sample Data via Git

Since the above method rarely works for anyone, there is a way to install the sample data from github. Create a directory anywhere outside your web root and clone the GitHub repo using this command:

git clone https://github.com/magento/magento2-sample-data.git
Clone the repo
Clone the repo

Next, run the installer script

php -f /dev/tools/build-sample-data.php -- --ce-source="/var/www/"

sample-data-2
After that finishes, go to your magento root, and run the updater:

bin/magento setup:upgrade

and you will see the sample data start to be installed. This process will take quite a while.
installing-data-now

Almost Finished

Clear the cache (and fix file permissions if needed) and you should see all the sample data in the front and back ends now.
Again, if you’re lucky, your store is ready to rock and roll.
You might have an issue where your css files are not loading.

CSS is missing
CSS is missing

From your web root, recreate the static content:

php bin/magento setup:static-content:deploy

static-content
Then, reindex

php bin/magento indexer:reindex
Reindexing Magento
Reindexing Magento

Check that the folder permissions are 755 in the pub folder and its subfolders and delete all the files and folders in var/cache, var/page_cache, and var/sessions and go reload your site.

Magento 2 Sample Data
Magento 2 Sample Data

Class Naming Conventions and the Autoloader

The way classes are named in Magento was taken from the Zend Framework, upon which Magento was developed.

Class names are standardized depending on their location on the file system. This standardization enables automatic class loading. The Magento autoloader is built in /app/Mage.php. Because class names are directly related to a specific path in the file system, the autoloader allows developers to include them without using include_once or require_once. A class named Mage_Page_Helper_Data, for example, would be found at /app/code/core/Mage/Page/Helper/Data.php