Composer – PHP Package Manager

The PHP Composer is a Package-Management Tool similar to NPM or Yarn which you can find in the JavaScript world of NodeJS.

Why do I need Dependency-Management?

There is already so much functional code present out there for e.g. User management inkl. roles, media management, login etc.

Therefore it wouldn’t be bad if you could just “import” these modules into your project and just use them, or not?

But as always in software development there are incompatibilities with specific versions.

And thats why we have composer.

Composer nows through the composer.json and composer.lock which modules need to be installed in which version and which modules are dependent on other modules.

Installation

https://getcomposer.org/download/

With there 4 lines of PHP you will get a composer.phar in the current directory.

Therefore you can execute composer with the following command:

php composer.phar list

But usually you have composer installed “globally”. This can only be done with admin/sudo privileges:

sudo mv composer.phar /usr/local/bin/composer

Due to the fact that every Linux and MacOS has the path /usr/local/bin/ in the $PATH variable you can now execute composer via:

composer list

Usage

Here are some essential composer commands explained:

composer list

Shows all available composer commands

composer create-project <boilerplate-name>

Creates a composer project with a defined “boilerplate” and creates a composer.json in the current directory.

composer require <modulenmae>

Adds the desired module to the composer.json and executes a composer update.

composer remove <modulenmae>

Removes the desired module from the composer.json and from the file system.

composer update

Updates all currently installed modules with the newest version (dependent on the version string defined in the composer.json)
The currently active version is saved in the composer.lock.

composer install

If a composer.lock is present all the modules are installed in the defined version.
If no composer.lock is present the given modules in the composer.json are installed in the newest version (according to composer.json) and a composer.lock is created.

composer self-update

Updates the composer version

composer outdated --direct

Shows all modules, which can be updated.
The –direct changes the output so only module updates are being displayed, which are defined in the composer.json and not the dependencies behind these modules.

Structure of the composer.json

The simplest composer.json can be created via the command composer init:

{
    "name": "devguide/myapp",
    "authors": [
        {
            "name": "Kevin Pfeifer",
            "email": "info@devguide.at"
        }
    ],
    "require": {}
}

Basically only the name of the module devguide/myapp, the author and the dependencies are being displayed here.

require vs. require-dev

As usual in software development there is a separation between the production and the development environment.

To exclude modules from the production environment you have to add these modules in the require-dev area. Modules, which are always required, should be in the require area.

composer install --no-dev

With this command the require-dev modules are not being installed.

Version strings for the composer.json

NameShort versionVersion Range
Exact Version1.0.21.0.2
Version Range>=1.0 <2.0>=1.0 <2.0
>=1.0 <1.1 || >=1.2>=1.0 <1.1 || >=1.2
Hyphenated Version Range1.0 – 2.0>=1.0.0 <2.1
1.0.0 – 2.1.0>=1.0.0 <=2.1.0
Wildcard Version Range1.0.*>=1.0 <1.1
Tile Version Range~1.2>=1.2 <2.0
~1.2.3>=1.2.3 <1.3.0
Caret Version Range^1.2.3>=1.2.3 <2.0.0

Composer Patches

As always in software development the current “stable” version doesn’t always work 100% as desired.

Before the Composer Module developer publishes a new version of the module you usually have access to patches.

These patch files are normal .patch files, which can be created via GIT.

Example: https://www.drupal.org/files/issues/2019-02-07/3030251—​entity-owner-trait—​11.patch

diff --git a/consumers.install b/consumers.install
index 3ca8e25..22287bc 100644
--- a/consumers.install
+++ b/consumers.install
@@ -8,7 +8,7 @@
 use Drupal\consumers\Entity\Consumer;
 use Drupal\Core\Field\BaseFieldDefinition;
 use Drupal\Core\StringTranslation\TranslatableMarkup;
-
+use Drupal\user\Entity\User;
 /**
  * Implements hook_install().
  */
@@ -161,3 +161,20 @@ function consumers_update_8105() {
     'is_default' => TRUE,
   ])->save();
 }
+
+/**
+ * 'owner_id' field should be not null.
+ */
+function consumers_update_8106() {
+  // Set owner_id to AnonymousUser id when null.
+  $anonymous_user = User::getAnonymousUser();
+
+  \Drupal::database()->update('consumer_field_data')
+    ->fields(['owner_id' => $anonymous_user->id()])
+    ->isNull('owner_id')
+    ->execute();
+
+  $entity_definition_update_manager = \Drupal::entityDefinitionUpdateManager();
+  $field_definition = $entity_definition_update_manager->getFieldStorageDefinition('owner_id', 'consumer');
+  $entity_definition_update_manager->updateFieldStorageDefinition($field_definition);
+}

Requirements

To automatically apply composer patches you need to do have the following done:

  1. “cweagans/composer-patches” needs to be required
  2. Enable patching via an option in the “extra” area
{
  "require": {
      "cweagans/composer-patches": "^1.6.0"
  },
  "extra": {
      "enable-patching": true
  }
}

Add patch to composer.jso

"patches": {
    "drupal/<module_or_theme>": {
        "<Description>": "<URL-to-patch>"
    }
},

Therefore

"patches": {
    "drupal/recaptcha": {
        "CAPTCHA validation error: unknown CAPTCHA session ID (issues/3035883)": "https://www.drupal.org/files/issues/2019-11-15/3035883-29-workaround.patch"
    }
},

In this example the module drupal/recaptcha will be patched.

If you execute a composer install or a composer update the path should show up in the output:

Problems with patching

In this example the given patch cant be downloaded because the URL is not available. Just check if the given URL is 100% correct.

In this example you can see that the patch can’t be applied.

There are several reasons why that can happen, but most of the time this happens, because a newer version of the module has already been installed which already includes the patch.

In that case check the changelog of the module if the given patch has already been applied and a new version has been published. If so just remove the patch from the composer.json

Problems while updating

Permission problems

PHP Memory Limit problems

The problem with the Memory Limit can be solved via the php.ini. First check where your php.ini is located:

php -i | grep 'Loaded Configuration File'

On MacOS this outputs (for me):

Loaded Configuration File => /etc/php.ini

On Linux

Loaded Configuration File => /etc/php/7.3/cli/php.ini

For Drupal 8 projects you should adjust the following value to at least 3G:

memory_limit = 3G

Permission problems always need to be checked dependent on the current environment.

While the update is running the web-site is NOT accessible and/or will output errors. If possible enable a maintenance mode of your web-site or stop the web server while updating is in progress.

Installed modules don’t match up with composer.json and/or composer.lock

Sometimes it can happen, that composer doesn’t download modules correctly or misses something.

The easiest solution is to delete all generated files/folders and just tell composer to install everything again.

rm -rf vendor/*
composer install

For Drupal 8 projects

rm -rf vendor/* docroot/modules/contrib/*
composer install

Where can I find composer modules?

All available composer modules can be found here: https://packagist.org/

What is PHP?

PHP (short for Hypertext Preprocessor) is an Open-Source scripting language which is mainly used to generate HTML-code when developing websites.

Example:

<?php
   $variable = "Testvariable";
   echo "This variable contains following text: " . $variable;
?>

Result:

This variable contains following text: Testvariable

In this example a basic variable is being defined and directly after that being outputted via “echo” after the text “This variable contains following text: “.

Due to the fact that PHP is a scripting language the content of the PHP file is being processed on every request. In contrast Java or C need to be “compiled” into machine code before it can be executed.

PHP offers quite a few functionalities out of the box – for example:

  • Database operations (MySQL, PostgreSQL, SQLite etc.)
  • Filesystem operations (Create/Edit/Delete folders or files etc.)
  • String manipulation (Create/Edit/Delete text etc.)
  • XML operations (Create/Edit/Delete XML data structures etc.)
  • and much more

These are just some basic which you can basically can find in any “Default PHP” installation. But PHP has so called “PHP modules” to extend the functionality PHP offers.

Popular PHP modules are:

  • XDebug (Extended Debug functionality)
  • OPCache (Saves precompiled bytecode from PHP in RAM instead of recompiling it on every request => performance boost)
  • MBString (Adds the ability to handle “MultiByte Strings” in PHP – e.g. Emoji Icons)
  • GD (Image editing in PHP like rotating or converting)
  • SOAP (A special, XML like data structure)
  • and much more

How PHP can be used via web servers or via the CLI can be seen in “CLI vs WebServer Integration“.

Due to the fact that PHP is always evolving I have gather the most important changes of PHP 7.1, 7.2 and 7.3 HERE.

Sources:
PHP: What is PHP – https://www.php.net/manual/de/intro-whatis.php

PHP versions

Everywhere in software development we have versions which evolve over time to improve performance, fix security issues and add new functionality. The following list contains a short overview of the changes from PHP 7.1 to 7.3.

PHP 5.6 and 7.0 will not be included here because they are no longer being supported (php.net) (April 2019)

PHP 7.1

“Nullable types”

function test1(?array $a) {
 
}
 
function test2($a): ?array {
 
}

The first function (test1) defines, that the first parameter can have the type “Array”, but can also be “null” (? before the array)

The second function (test2) defines, that the return value can have the type “Array”, but also “null” too.

Without the prefixed ? the function test1 called with a parameter “null” or the second function test2 with a return value of “null” would lead into a “Fatal Error” when executing that code.

Array and list have same functionality

Old Code:

$array = array(0 => 'a', 1 => 'b');
 
list($a, $b) = $array;
 
// $a is = 'a'
// $b is = 'b'

New Code:

$array = array(0 => 'a', 1 => 'b', 2 => 'c');
 
[$a, $b] = $array;
 
// $a is = 'a'
// $b is = 'b'
 

list(1 => $b, 2 => $c) = $array;
// same as above
[1 => $b, 2 => $c] = $array;

Visibility of constants in classes

As already common in other object oriented programming languages you can now set “visibilities” for class constants.

  • public Access to variable allowed from everywhere.
  • protected Access to variable only allowed in its own class AND all extended classes.
  • private Access to variable only allowed in its own class.
class test {
    const PUBLIC1 = 'TEST';
 
    public const PUBLIC2 = 'TEST';
    protected const PROTECTED = 'TEST';
    private const PRIVATE = 'TEST';
}

Multi catch exception handling

You can now have multiple exceptions in one try-catch block.

try {
     throw new Exception('Fehler');
} catch (Exception | AndererExceptionTyp $catchedException) {
     var_dump($catchedException);
}

mcrypt extension deprecated => use OpenSSL

The functions of the “mcrypt” extension (function names all start with “mcrypt_”) will be marked as “deprecated” markiert and therefore produce a line in the error.log.

Use the OpenSSL extension as a replacement.

PHP 7.2

Native support for BMP image format

Since PHP 7.2 the GD extension allows to handle .bmp images.

Typ “object” bei Parameter- und Rückgabewerten einstellbar

A new type, object, has been introduced that can be used for parameter typing and return typing of any objects.

function test (object $a): object {

}

Enhancements to the EXIF extension

EXIF (Exchangeable Image File Format) is a standard to save metadata in image files.

Till now the automatically parsed EXIFF data for images files was very limited. With PHP 7.2 many EXIF formats from well known camera suppliers have been added. See HERE

Encrypted ZIP-Archives

With PHP 7.2 you can now create ZIP archives with a password protection.

PHP 7.3

Trailing Commas are allowed in Calls

With PHP 7.3. it is now allowed to have a trailing comma even if it is the last parameter in a function call.

my_function(
    $param1,
    $param2,
);

JSON_THROW_ON_ERROR

Till now json_decode() returns null if an error occurs.
But null can also be a valid result, which can lead to confusion.

So now you can check for json errors with these functions:

  • json_last_error()
    • Returns (if present) the last error, which has occurred on the las encoding/decoding process for a JSON.
  • json_last_error_msg()
    • Return “No error” if the encode /decode was a success and FALSE if there were problems.

An anonymous user has written a very nice helper function on the json_last_error_msg() definition page on php.net:

<?php
    if (!function_exists('json_last_error_msg')) {
        function json_last_error_msg() {
            static $ERRORS = array(
                JSON_ERROR_NONE => 'No error',
                JSON_ERROR_DEPTH => 'Maximum stack depth exceeded',
                JSON_ERROR_STATE_MISMATCH => 'State mismatch (invalid or malformed JSON)',
                JSON_ERROR_CTRL_CHAR => 'Control character error, possibly incorrectly encoded',
                JSON_ERROR_SYNTAX => 'Syntax error',
                JSON_ERROR_UTF8 => 'Malformed UTF-8 characters, possibly incorrectly encoded'
            );

            $error = json_last_error();
            return isset($ERRORS[$error]) ? $ERRORS[$error] : 'Unknown error';
        }
    }
?>

Alternatively you an also solve that problem with a try-catch block:

try {
    json_decode("{", false, 512, JSON_THROW_ON_ERROR);
}
catch (\JsonException $exception) {
    echo $exception->getMessage(); // echoes "Syntax error"
}

is_countable function

Currently it is common to check a variable if it can be looped through a foreach with the following code:

$array = [1,2,3,4,5,6,7,8];

if(is_array($array) && sizeof($array) > 0){
  foreach($array as $value){
    
  }
}

But with PHP 7.3 you can write:

$array = [1,2,3,4,5,6,7,8];

if(is_countable($array)){
  foreach($array as $value){
    
  }
}

array_key_first(), array_key_last()

Till now it was not “easy” to get the first and last “key” of an array. But with PHP 7.3 you now have the functions array_key_first()and array_key_last().

Sources

mod_php vs (Fast)CGI vs FPM

There are different ways how a web server can handle PHP files. Following are the most common implementations how this is done:

mod_php

“mod_php” is a module for the web-server “Apache”.

With this module PHP is therefore “integratedinto the web-server. So there is no extra PHP-Process which handles the PHP interpretation. Instead everything is handled by the Apache process.

The main advantage for using “mod_php” is performance. Compared to CGI you usually gain 300-500% when changing to mod_php.

Main reason for that is the ability to cache PHP modules, or specifically the configuration which usually (in CGI and FastCGI) has to be parsed on every request.

CGI

The “Common Gateway Interface” (short CGI) implementation means, that the web-server starts an extra PHP process for each request. Therefore all PHP modules, the php.ini and all other configuration needs to be loaded and parsed on every request, which is inefficient.

Main advantage for using CGI is the complete separation between the executing web-server Code and the PHP code.

FastCGI

FastCGI is a PHP implementation, which contains the security advantages from CGI but also being efficient like mod_php.

Here we don’t start a new PHP process on every request, instead we use “ready made” PHP interpreter instances which only get the PHP files passed on to be handled.

FPM

The „PHP-FastCGI Process Manager“ (short PHP-FPM) is an alternative to the FastCGI implementation.

Here the main difference is, that there is always a “parallel” PHP-FPM Process which is connected to the web-server process.

For more details on FPM see HERE

Source: https://blog.layershift.com/which-php-mode-apache-vs-cgi-vs-fastcgi/

PHP-FPM

The “PHP-FastCGI Process Manager” (short PHP-FPM) is an alternative FastCGI Implementation for PHP in a web-server.

Here we always have (at least) one PHP-FPM process parallel to the web-server process which handles PHP interpretation.

FPM processes group up into different “pools”. In these pools there will usually be several proccess created which handle PHP interpretation for a specific web page.

The amount of these processes and many more settings can be set in the FPM configuration (see bellow).

Installation (Debian based Distros)

sudo apt install php7.2 php7.2-common php7.2-cli php7.2-fpm

FPM Configuration

This is located in “/etc/php/7.2/fpm/pool.d/”.
Here we should create one file per socket/website.

[<pool-name>]
user = <username>
group = <groupname>
listen = /run/php/<socket-name>.sock
listen.owner = www-data
listen.group = www-data
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
chroot = <docroot-path>
php_admin_value[openssl.capath] = /etc/ssl/certs

The “listen” defines where the linux socket should be placed in the file system. This socket will be created after the PHP-FPM process has been restarted.

Via this socket the web-server can redirect PHP interpretation to the FPM process (see bellow)

Web-Server Configuration (NGINX)

The web-server has to know how and where it should send PHP-Files to the corresponding PHP process.

This is an example for: /etc/nginx/sites-available/<domain-name>.conf

server {
    listen 443 ssl http2; # managed by Certbot
    listen [::]:443 ssl http2;

    ssl_certificate /etc/letsencrypt/live/<domain-name>/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/<domain-name>/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    root <path-to-docroot>;
    server_name <domain-name>;

    index index.html index.htm index.php;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }	

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
	fastcgi_pass unix:/run/php/<fpm-socket-name>.sock;
    }
	
    error_log <path-to-logs>/error.log warn;
    access_log <path-to-logs>/access.log apm;
}

server {
    if ($host = <domain-name>) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
	
    listen 80;
    listen [::]:80;

    server_name <domain-name>;
    return 404; # managed by Certbot
}

The following line is responsible for the “redirection” to the FPM process:

fastcgi_pass unix:/run/php/<fpm-socket-name>.sock;

When you change something in the PHP-FPM config or the NGINX config you should always restart both services:

sudo systemctl restart php7.2-fpm.service nginx

CLI vs web-server integration

As described in PHP-FPM you can have different configurations for different web pages.

One advantage for that is the fact, that you can have different PHP versions for different web pages.

But these configurations don’t change the system-wide installed PHP version.

This means, that the PHP version from a PHP-Info file doesn’t have the be the same coming from the web-server and coming from the CLI.

php -v
PHP 7.2.19-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: May 31 2019 11:17:15) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.19-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.7.1, Copyright (c) 2002-2019, by Derick Rethans

How to change the system-wide default PHP version?

Note: This is only possible if you have root/sudo privileges on the server!

Via the following command you can find out where the php command is located:

which php

On my server this is:

/usr/bin/php

If you now check this “file” you see:

ls -al /usr/bin/php        
lrwxrwxrwx 1 root root 21 Jan  3  2018 /usr/bin/php -> /etc/alternatives/php

This means, that /usr/bin/php only points to /etc/alternatives/php. But as you can see on the path this is handled by the “alternatives”.

If you check this “file” you see:

ls -al /etc/alternatives/php        
lrwxrwxrwx 1 root root 15 May 31  2018 /etc/alternatives/php -> /usr/bin/php7.2

This now points to the “real” binary, which executes PHP.

ls -al /usr/bin/php7.2      
-rwxr-xr-x 1 root root 4899864 May 31 13:17 /usr/bin/php7.2

Therefore the command “php” points to “/usr/bin/php”, which points to “/etc/alternatives/php” and finally to “/usr/bin/php7.2”.

To change this “link” (and therefore change the CLI PHP version) you can execute the following command:

sudo update-alternatives --set php /usr/bin/php7.3

After that you successfully changed the PHP CLI from 7.2 to 7.3.

php -v
PHP 7.3.6-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: May 31 2019 11:06:48) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.6, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.6-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.7.1, Copyright (c) 2002-2019, by Derick Rethans