Naguel

I'm Nahuel, and these are my work experiences, ideas and thoughts as a web developer working on the eCommerce industry.

Proper way to configure asynchronous email sending in Magento

Proper way to configure asynchronous email sending in Magento

There are two ways to configure how Magento send order related emails (order confirmation, invoice, shipment and credit memo emails):

  • Either immediately when an action is performed (for example, right after you place an order).
  • Or cron-based (asynchronous).

Magento itself recommends to configure this to be asynchronous for performance reasons, so the email dispatch process doesn't block nor delay what the customer is doing (for example, less time consumption when the customer is placing an order), or to avoid long cron runs (or total crons block) when an ERP is updating orders status in bulk.

Enabling the “Asynchronous email notifications” setting moves processes that handle checkout and order processing email notifications to the background.

Configuration best practices by Magento

For my personal experience, it is also recommended to configure emails sending to be asynchronous to avoid having unsent emails.

When this is configured as immediately (not async), if an email fails to be dispatched because of any reason (like a server timeout) then Magento won’t try to send that email again in the future and the customer won’t be ever notified of his/her action.

Basically, there are a lot of good reasons to have this to be async.

Before setting this up

If you are going live for the first time on a new site there's not much to worry about, but if you are changing this setting from not-async to cron-based on an already live site then there's something to consider first.

The way this works when it's configured to be async is that Magento will rely on a cron to pick up from the database (from those tables related to Orders, Invoices, Shipments and Credit Memos) all those entities where the email was not dispatched, and it will send it.

Problem is that Magento doesn't care much about the date of those entities.

For example, if there’s an Order from 2019 whose order confirmation email was not sent because of any reason (for example, because there was a PHP error that stopped the execution), the cron will now pick this Order up and Magento will send the email related to that order.

To avoid this behaviour we should mark all pending emails from Orders, Invoices, Shipments, and Credit Memos as if they doesn't need to be dispatched, which can be done by executing the following SQL statements before enabling the asynchronous sending.

UPDATE sales_order SET send_email = 0;
UPDATE sales_invoice SET send_email = 0;
UPDATE sales_shipment SET send_email = 0;
UPDATE sales_creditmemo	 SET send_email = 0;

What we are doing here is ignoring the sending of emails for old Orders (and Invoices, Shipments and Credit Memos) so when we change this to be async the cron will only cares for future stuff.

Changing the configuration

Enable the “Asynchronous sending” option under “Stores → Settings → All Stores → Sales → Sales Email → General Settings”.

Magento and a deadlock found when trying to get lock

Magento and a deadlock found when trying to get lock

"What now?" issues every now and then with Magento are the norm, and I got this one a few days ago:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, query was: INSERT INTO `catalog_product_index_eav_temp` SELECT DISTINCT [...]

Apparently the server went down for a second, or MySQL went down... or something happened and a reindex process was interrupted (in my case the one for the "Product EAV" index), and there was no way to make it work again.

Even the usual bin/magento indexer:reset and then a manual reindex with bin/magento indexer:reindex didn't do the trick and the only difference was that instead of the previous error I was getting the classic one.

Product EAV index is locked by another reindex process. Skipping.

Lucky me, I find the following on Twitter.

Basically, when you have no way to "unlock" an index, and after you already tried the reset and manual reindex, just change that prefix in your app/etc/env.php file to something new, then reset and finally try to reindex again.

That alone should do the trick.

Install Elasticsearch for Magento 2 on macOS

Install Elasticsearch for Magento 2 on macOS

Depending on the Magento version you have, or the Elasticsearch version you want, the process of getting this work locally can be easy or a nightmare I'll try to simplify here in this post.

If you already know what version to install, skip the following and jump straight to the installation instructions.

What Elasticsearch version do you need?

If we dive into the official Magento documentation and take a quick look at the different changes made to it across time, we can learn a few things.

Magento 2.3.4 documentation

Available at the magento/devdocs' GitHub repo, tag 2.3.4.

Magento 2.3.1 adds support for Elasticsearch 6.x and it is enabled by default. Magento still provides connectivity for Elasticsearch 2.x and 5.x but these must be enabled in order to use these versions.

Magento 2.3.5 documentation

Available at the magento/devdocs' GitHub repo, tag 2.3.5.

Magento 2.3.5 adds support for Elasticsearch 7.x.x (default) and 6.8.x. Both ES 2.x and 5.x are End of Life and are no longer supported in Magento.

Magento 2.4 documentation

Available at the magento/devdocs' GitHub repo, branch 2.4.1-develop.

You must install and configure Elasticsearch 7.6.x before upgrading to Magento 2.4.0.
Magento does not support Elasticsearch 2.x, 5.x, and 6.x.

Or, to summarize:

  • For Magento 2.3.1 to Magento 2.3.4 you need to install Elasticsearch 6.x.
  • For Magento 2.3.5 to Magento 2.3.n (anything before 2.4.0) you'll need to install Elasticsearch 7.x.x. It works also with 6.8.x, but version 7.x.x is easy to install.
  • For Magento 2.4.0 and up you need to install Elasticsearch 7.6.x. No support for previous versions.

If you are working with multiple Magento projects on different versions, I would suggest you to install Elasticsearch 6.x to support from Magento 2.3.1 to Magento 2.3.5, and everything before Magento 2.4.

If you have all projects on Magento 2.3.5 and up then install Elasticsearch 7.6.x.

Install Elasticsearch 6.x on macOS

You need Homebrew first, so please install it by doing the following:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

You can check the official Homebrew page for more information about it and alternative installations options.

Then, we need to add a third-party repo to Homebrew and a few packages, including Elasticsearch itself.

brew tap elastic/tap; brew cask install homebrew/cask-versions/adoptopenjdk8; brew install elasticsearch@6.8;

When you do that you'll get a lot of output on your terminal, but you need to only care for the installation path of the last elasticsearch@6.8 package.

➜  brew install elasticsearch@6.8;
==> Downloading https://homebrew.bintray.com/bottles/elasticsearch%406.8-6.8.8.catalina.bottle.tar.gz
Already downloaded: /Users/nahuelsanchez/Library/Caches/Homebrew/downloads/01d0782011cbecdb3b1125469ee2ed60ef07926a8f36752c64d1b8c2b763736d--elasticsearch@6.8-6.8.8.catalina.bottle.tar.gz
==> Pouring elasticsearch@6.8-6.8.8.catalina.bottle.tar.gz
==> /usr/local/Cellar/elasticsearch@6.8/6.8.8/bin/elasticsearch-keystore create
==> Caveats
Data: /usr/local/var/lib/elasticsearch/
Logs: /usr/local/var/log/elasticsearch/elasticsearch_nahuelsanchez.log
Plugins: /usr/local/var/elasticsearch/plugins/
Config: /usr/local/etc/elasticsearch/

elasticsearch@6.8 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have elasticsearch@6.8 first in your PATH run:
  echo 'export PATH="/usr/local/opt/elasticsearch@6.8/bin:$PATH"' >> ~/.zshrc
  
To have launchd start elasticsearch@6.8 now and restart at login:
  brew services start elasticsearch@6.8
Or, if you don't want/need a background service you can just run:
  elasticsearch
==> Summary
🍺  /usr/local/Cellar/elasticsearch@6.8/6.8.8: 136 files, 103.4MB

The path below "Summary" is the one we need to mind, where mine is /usr/local/Cellar/elasticsearch@6.8/6.8.8.

Go to that folder and install a few Elasticsearch plugins.

bin/elasticsearch-plugin install analysis-phonetic bin/elasticsearch-plugin install analysis-icu; bin/elasticsearch-plugin install analysis-smartcn;

Now, because only God knows why, we need to fix something on this freshly installed Elasticsearch package, otherwise it won't start.

Standing again into that installation path, cd into the libexec/config folders, and edit the jvm.options file.

Comment the line it says 8:-Xloggc:logs/gc.log by adding a # at the beginning of it, and below add 8:-Xloggc:/tmp/logs_gc.log.

This is how the file might ended up looking

You are basically done now.

Start Elasticsearch by doing brew services start elasticsearch@6.8.

➜  brew services start elasticsearch@6.8
==> Successfully started `elasticsearch@6.8` (label: homebrew.mxcl.elasticsearch@6.8)

To check if everything is good, you can do a cURL to the Elasticsearch instance running.

➜  curl "http://localhost:9200/_nodes/settings?pretty=true"
{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch_brew",
  "nodes" : {
    "ArBWW_c3QRunhfQn460sxQ" : {
      "name" : "ArBWW_c",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1",
      "version" : "6.8.8",
      "build_flavor" : "oss",
      "build_type" : "tar",
      "build_hash" : "2f4c224",
      "roles" : [
        "master",
        "data",
        "ingest"
      ],
      "settings" : {
        "client" : {
          "type" : "node"
        },
        "cluster" : {
          "name" : "elasticsearch_brew"
        },
        "http" : {
          "type" : {
            "default" : "netty4"
          }
        },
        "node" : {
          "name" : "ArBWW_c"
        },
        "path" : {
          "data" : [
            "/usr/local/var/lib/elasticsearch"
          ],
          "logs" : "/usr/local/var/log/elasticsearch",
          "home" : "/usr/local/Cellar/elasticsearch@6.8/6.8.8/libexec"
        },
        "transport" : {
          "type" : {
            "default" : "netty4"
          }
        }
      }
    }
  }
}

That host and port on that cURL command (which by default is localhost:9200) is the host and port you need to use for configuring Elasticsearch in Magento. If it doesn't work, try the same cURL call but with 127.0.0.1:9200.

Remember to brew services stop elasticsearch@6.8 when you are done working so it's not running forever in your computer.

Install Elasticsearch 7.x on macOS

You need Homebrew first, so please install it by doing the following:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

You can check the official Homebrew page for more information about it and alternative installations options.

Then, we need to add a third-party repo to Homebrew and the Elasticsearch package itself.

brew tap elastic/tap; brew install elastic/tap/elasticsearch-full;

When you do that you'll get a lot of output on your terminal, but you need to only care for the installation path of the elastic/tap/elasticsearch-full package, which is located just at the end below "Summary".

For example, mine says /usr/local/Cellar/elasticsearch-full/7.8.0.

Go to that folder and install a few Elasticsearch plugins.

bin/elasticsearch-plugin install analysis-phonetic bin/elasticsearch-plugin install analysis-icu; bin/elasticsearch-plugin install analysis-smartcn;

That's all, basically.

Start Elasticsearch by doing brew services start elastic/tap/elasticsearch-full.

To check if everything is good, you can do a cURL to the Elasticsearch instance by running curl "http://localhost:9200/_nodes/settings?pretty=true" which will output a big JSON.

That host and port on that cURL command (which by default is localhost:9200) is the host and port you need to use for configuring Elasticsearch in Magento. If it doesn't work, try the same cURL call but with 127.0.0.1:9200.

Remember to brew services stop elastic/tap/elasticsearch-full when you are done working so it's not running forever in your computer.

Can I have both installed at the same time?

No... well, I didn't try it but I think you can't. I would avoid such experiment.

What you can do for sure is uninstall one and install the other on demand.

If you do that you need to uninstall the Elasticsearch plugins you install it and install them again so you ended up with the right plugins version compatible with the Elasticsearch version currently installed.

Anything else?

Reindex Magento before trying to load the store.

Magento virtual themes or why theme changes don't show up in the front end

Magento virtual themes or why theme changes don't show up in the front end

You have a Magento 2 theme created (at least you can see it in your IDE and we're going to assume there's no typo in the theme.xml nor in the registration.php), you can also see it in the Admin, and finally you have it configured for your store.

Then you go to the front end and you see a Frankenstein's monster: the CSS is okay but the Layout XML is not being taken into consideration. The front end kind of knows which theme to load, but it's loading it only partially.

What it's going on? How do we fix it?

One of the possibilities is that the theme is set as Virtual instead of Physical in the database, causing behaviors such as the described above.

Magento 2 happen to have 3 types of themes, of which I have found very little information in the official documentation (just a minor mention in the Troubleshooting section of the 2.0 "Apply and configure a storefront theme" page but gone in the 2.2 version of it) but a convincing explanation in this StackExchange question.

We have "Physical themes" which are the well-known themes such as Blank and Luma, second we have "Virtual themes" which appears to be themes that extend a Physical theme but doesn't exist in the files (nothing in app/design), and finally we have the "Staging themes" which I have no idea what they are nor how they behave (sorry).

These 3 different types of themes are defined in the Interface Magento\Framework\View\Design\ThemeInterface as 0, 1, and 2 for Physical, Virtual and Staging, respectively.

If you go to your database (using the Terminal, PhpMyAdmin or an app like TablePlus) and take a look at the theme table, chances are that you'll find something like this:

If you encounter a 1 in the type column for your theme, there's your problem. Change it to 0 and everything will be fixed.

How the theme became Virtual without anybody setting it anywhere?

You're probably wondering how that 1 ended up there without anybody setting it (at least you're sure you didn't as you just find out about all this).

As described, Virtual themes are themes that doesn't exist in the code, meaning there are no files for that theme (how Magento expect us to use this type of theme is for me unknown).

If you're using a version control system such as Git maybe (maybe) you ended up for a moment in a branch without the theme files and somehow loaded the front end or ran a bin/magento command. For a moment only, Magento saw in the database that a theme is set and active but found no files in the code for such theme, so it changed it to Virtual.

Install Composer on OS X and use it globally

Install Composer on OS X and use it globally

Magento 2 needs Composer, so let install it on OS X in a way it can be used globally.

Considering that installing Composer will create a composer.phar file, and our intention is to use it everywhere, anytime, regardless the project we're working on, I decided to install it (meaning let the composer.phar be created) in /Applications/Composer (it makes sense to me, but you can choose the destination folder you want).

Installing Composer

First thing first, let's go to /Applications, create the Composer folder, and get in.

cd /Applications/
mkdir Composer
cd Composer

Now, we can install Composer for real using cURL.

curl -sS https://getcomposer.org/installer | php

If everything goes as expected we'll get a nice message saying something like "Composer successfully installed to..." and we can proceed to the next step.

Making it globally available

Now that we have the composer.phar file, we need (or want?) to have it globally available by typing composer in the command line. To achieve this let's start by moving our composer.phar file to /usr/bin/.

sudo mv composer.phar /usr/bin/

And finally let's create an alias on our bash profile.

alias composer="php /usr/bin/composer.phar"

Reload the command line profile, and that's all it takes to get Composer on OS X. You can now go to any folder you want and type composer about. If something nice appears it means everything is okay.

I might be skipping the part where I'm supposed to explain how to edit and reload the bash profile. It was on purpose because it always different depending on the terminal you use (for example mines it's located in ~/.zshrc because I use Oh My ZSH!).

You can try the following in order to access and edit the bash profile.

vim ~/.bash_profile