DI Container Benchmark

Author:Máté Kocsis (@kocsismate90)
Repository:https://github.com/kocsismate/php-di-container-benchmarks
Generated:2019-01-13 13:49:11

Table of Contents


Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

Name Version Compiled/Dynamic Autowiring
aura/di 3.4.0 dynamic supported
rdlowrey/auryn v1.4.2 dynamic supported
jshannon63/cobalt v1.2.1 dynamic supported
level-2/dice 4.0 dynamic supported
bitexpert/disco v0.10.0 compiled not supported
joomla/di 1.5.1 dynamic supported
illuminate/container v5.7.20 dynamic supported
opulence/ioc v1.1.5 dynamic supported
php-di/php-di 6.0.5 compiled supported
phpixie/di 3.2.1 dynamic not supported
pimple/pimple v3.2.3 dynamic not supported
symfony/dependency-injection v4.2.2 compiled supported
thecodingmachine/yaco v1.0.0 compiled not supported
yiisoft/yii2 2.0.15.1 dynamic supported
woohoolabs/zen 2.7.1 compiled supported
zendframework/zend-servicemanager 3.4.0 dynamic not supported

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.


Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (10 or 100 objects). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Suites: "Cold" ones (Test Suite 1-2) measure performance including autoloading and startup time of containers as well as autoloading time of the retrieved objects. "Semi-Warm" ones (Test Suite 3-4) measure performance excluding container autoloading time, but including startup time and autoloading time of the retrieved objects, while "Warm" ones (Test Suite 5-6) exclude autoloading and startup time equally. Time of compilation is always excluded from the results due to OPcache.

Each Test Suite contains three Test Cases which define the number of iterations the main task has to be repeated in order to simulate real world usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results are displayed in the final results.


Setup

The benchmark is run on AWS EC2, using a C5.large instance. The operating system on the host is Ubuntu 18.04. PHP 7.3 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.


Results

Test Suite 1: "Cold" Retrieval of a small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.037 100% 0.524 100%
2 Symfony 0.052 141% 0.531 101%
3 Yaco 0.072 195% 0.563 107%
4 PHP-DI 0.078 211% 0.53 101%
5 PHPixie DI 0.086 232% 0.565 108%
6 Joomla 0.086 232% 0.599 114%
7 Dice 0.086 232% 0.554 106%
8 Zend ServiceManager 0.087 235% 0.566 108%
9 Cobalt 0.105 284% 0.551 105%
10 Pimple 0.109 295% 0.571 109%
11 Laravel 0.112 303% 0.544 104%
12 Opulence IoC 0.121 327% 0.553 106%
13 Yii2 Container 0.131 354% 0.601 115%
14 Auryn 0.174 470% 0.56 107%
15 Aura 0.194 524% 0.579 111%
16 Disco 0.315 851% 0.635 121%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.042 100% 0.524 100%
2 Symfony 0.056 133% 0.531 101%
3 Yaco 0.078 186% 0.563 107%
4 PHP-DI 0.084 200% 0.53 101%
5 Dice 0.094 224% 0.554 106%
6 Zend ServiceManager 0.095 226% 0.566 108%
7 Joomla 0.1 238% 0.599 114%
8 PHPixie DI 0.108 257% 0.565 108%
9 Pimple 0.109 260% 0.571 109%
10 Cobalt 0.112 267% 0.551 105%
11 Laravel 0.135 321% 0.544 104%
12 Yii2 Container 0.137 326% 0.601 115%
13 Opulence IoC 0.139 331% 0.553 106%
14 Aura 0.198 471% 0.579 111%
15 Auryn 0.212 505% 0.56 107%
16 Disco 0.335 798% 0.635 121%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.087 100% 0.524 100%
2 Symfony 0.101 116% 0.531 101%
3 Yaco 0.128 147% 0.563 107%
4 PHP-DI 0.136 156% 0.53 101%
5 Zend ServiceManager 0.147 169% 0.566 108%
6 Dice 0.158 182% 0.554 106%
7 Yii2 Container 0.19 218% 0.601 115%
8 Cobalt 0.196 225% 0.551 105%
9 Pimple 0.222 255% 0.571 109%
10 Joomla 0.228 262% 0.599 114%
11 Aura 0.251 289% 0.579 111%
12 Opulence IoC 0.309 355% 0.553 106%
13 PHPixie DI 0.336 386% 0.565 108%
14 Laravel 0.343 394% 0.544 104%
15 Disco 0.535 615% 0.635 121%
16 Auryn 0.581 668% 0.56 107%

Test Suite 2: "Cold" Retrieval of a big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.15 100% 0.64 100%
2 Zen 0.225 150% 0.622 97%
3 Yaco 0.27 180% 0.67 105%
4 PHPixie DI 0.289 193% 0.663 104%
5 Joomla 0.299 199% 0.706 110%
6 PHP-DI 0.308 205% 0.648 101%
7 Zend ServiceManager 0.318 212% 0.681 106%
8 Pimple 0.352 235% 0.686 107%
9 Opulence IoC 0.509 339% 0.753 118%
10 Cobalt 0.529 353% 0.863 135%
11 Yii2 Container 0.543 362% 0.822 128%
12 Laravel 0.546 364% 0.718 112%
13 Dice 0.559 373% 0.944 147%
14 Disco 0.57 380% 0.767 120%
15 Aura 0.719 479% 0.894 140%
16 Auryn 0.836 557% 0.828 129%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.157 100% 0.64 100%
2 Zen 0.232 148% 0.622 97%
3 Yaco 0.278 177% 0.67 105%
4 PHPixie DI 0.311 198% 0.663 104%
5 Joomla 0.314 200% 0.706 110%
6 PHP-DI 0.314 200% 0.648 101%
7 Zend ServiceManager 0.323 206% 0.681 106%
8 Pimple 0.362 231% 0.686 107%
9 Opulence IoC 0.531 338% 0.753 118%
10 Cobalt 0.54 344% 0.863 135%
11 Yii2 Container 0.551 351% 0.822 128%
12 Laravel 0.566 361% 0.718 112%
13 Dice 0.571 364% 0.944 147%
14 Disco 0.603 384% 0.767 120%
15 Aura 0.728 464% 0.894 140%
16 Auryn 0.875 557% 0.828 129%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.204 100% 0.64 100%
2 Zen 0.275 135% 0.622 97%
3 Yaco 0.329 161% 0.67 105%
4 PHP-DI 0.366 179% 0.648 101%
5 Zend ServiceManager 0.378 185% 0.681 106%
6 Joomla 0.444 218% 0.706 110%
7 Pimple 0.478 234% 0.686 107%
8 PHPixie DI 0.544 267% 0.663 104%
9 Yii2 Container 0.608 298% 0.822 128%
10 Cobalt 0.613 300% 0.863 135%
11 Dice 0.628 308% 0.944 147%
12 Opulence IoC 0.69 338% 0.753 118%
13 Laravel 0.778 381% 0.718 112%
14 Aura 0.787 386% 0.894 140%
15 Disco 0.796 390% 0.767 120%
16 Auryn 1.255 615% 0.828 129%

Test Suite 3: "Semi-Warm" Instantiation of a small object graph (10 objects)

In this Test Suite, containers have to create an object graph of 10 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.03 100% 0.531 100%
2 Zen 0.032 107% 0.524 99%
3 PHPixie DI 0.072 240% 0.603 114%
4 Disco 0.072 240% 0.632 119%
5 Joomla 0.077 257% 0.673 127%
6 Cobalt 0.078 260% 0.559 105%
7 Zend ServiceManager 0.086 287% 0.604 114%
8 Dice 0.108 360% 0.545 103%
9 Pimple 0.13 433% 0.633 119%
10 Yii2 Container 0.153 510% 0.627 118%
11 Opulence IoC 0.169 563% 0.555 105%
12 Aura 0.174 580% 0.554 104%
13 Laravel 0.275 917% 0.548 103%
14 Auryn 0.349 1163% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.085 100% 0.531 100%
2 Zen 0.089 105% 0.524 99%
3 Disco 0.204 240% 0.632 119%
4 Cobalt 0.257 302% 0.559 105%
5 PHPixie DI 0.269 316% 0.603 114%
6 Joomla 0.279 328% 0.673 127%
7 Zend ServiceManager 0.383 451% 0.604 114%
8 Dice 0.441 519% 0.545 103%
9 Pimple 0.618 727% 0.633 119%
10 Yii2 Container 0.727 855% 0.627 118%
11 Aura 0.804 946% 0.554 104%
12 Opulence IoC 0.988 1162% 0.555 105%
13 Laravel 2.136 2513% 0.548 103%
14 Auryn 2.657 3126% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.637 100% 0.531 100%
2 Zen 0.657 103% 0.524 99%
3 Disco 1.539 242% 0.632 119%
4 Cobalt 2.062 324% 0.559 105%
5 PHPixie DI 2.272 357% 0.603 114%
6 Joomla 2.32 364% 0.673 127%
7 Zend ServiceManager 3.362 528% 0.604 114%
8 Dice 3.788 595% 0.545 103%
9 Pimple 5.541 870% 0.633 119%
10 Yii2 Container 6.479 1017% 0.627 118%
11 Aura 7.097 1114% 0.554 104%
12 Opulence IoC 9.174 1440% 0.555 105%
13 Laravel 20.662 3244% 0.548 103%
14 Auryn 25.7 4035% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 4: "Semi-Warm" Instantiation of a big object graph (100 objects)

In this Test Suite, containers have to create an object graph of 100 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.165 100% 0.636 100%
2 Zen 0.267 162% 0.622 98%
3 Disco 0.424 257% 0.728 115%
4 PHPixie DI 0.44 267% 0.703 110%
5 Joomla 0.485 294% 0.771 121%
6 Zend ServiceManager 0.573 347% 0.708 111%
7 Cobalt 0.654 396% 0.9 141%
8 Pimple 0.816 495% 0.679 107%
9 Dice 0.852 516% 0.856 135%
10 Yii2 Container 1.101 667% 0.814 128%
11 Aura 1.279 775% 0.869 137%
12 Opulence IoC 1.316 798% 0.752 118%
13 Laravel 2.357 1428% 0.729 115%
14 Auryn 3.183 1929% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.629 100% 0.636 100%
2 Zen 0.74 118% 0.622 98%
3 Disco 1.662 264% 0.728 115%
4 PHPixie DI 2.263 360% 0.703 110%
5 Cobalt 2.419 385% 0.9 141%
6 Joomla 2.618 416% 0.771 121%
7 Zend ServiceManager 3.447 548% 0.708 111%
8 Dice 4.302 684% 0.856 135%
9 Pimple 5.774 918% 0.679 107%
10 Yii2 Container 6.839 1087% 0.814 128%
11 Aura 7.62 1211% 0.869 137%
12 Opulence IoC 9.888 1572% 0.752 118%
13 Laravel 20.667 3286% 0.729 115%
14 Auryn 26.954 4285% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 5.251 100% 0.636 100%
2 Zen 5.516 105% 0.622 98%
3 Disco 14.07 268% 0.728 115%
4 Cobalt 19.953 380% 0.9 141%
5 PHPixie DI 20.344 387% 0.703 110%
6 Joomla 23.8 453% 0.771 121%
7 Zend ServiceManager 32.13 612% 0.708 111%
8 Dice 38.613 735% 0.856 135%
9 Pimple 54.691 1042% 0.679 107%
10 Yii2 Container 64.233 1223% 0.814 128%
11 Aura 70.978 1352% 0.869 137%
12 Opulence IoC 95.545 1820% 0.752 118%
13 Laravel 202.686 3860% 0.729 115%
14 Auryn 264.408 5035% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 5: "Warm" Fetching of the same small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.051 100% 0.524 100%
2 Symfony 0.053 104% 0.531 101%
3 Yaco 0.06 118% 0.563 107%
4 PHP-DI 0.061 120% 0.53 101%
5 Zend ServiceManager 0.062 122% 0.566 108%
6 Aura 0.063 124% 0.579 111%
7 Yii2 Container 0.067 131% 0.601 115%
8 Dice 0.07 137% 0.554 106%
9 Cobalt 0.094 184% 0.551 105%
10 Pimple 0.118 231% 0.571 109%
11 Joomla 0.142 278% 0.599 114%
12 Opulence IoC 0.179 351% 0.553 106%
13 Disco 0.22 431% 0.635 121%
14 Laravel 0.231 453% 0.544 104%
15 PHPixie DI 0.254 498% 0.565 108%
16 Auryn 0.405 794% 0.56 107%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.503 100% 0.524 100%
2 Symfony 0.525 104% 0.531 101%
3 Yaco 0.592 118% 0.563 107%
4 PHP-DI 0.599 119% 0.53 101%
5 Zend ServiceManager 0.612 122% 0.566 108%
6 Aura 0.625 124% 0.579 111%
7 Yii2 Container 0.664 132% 0.601 115%
8 Dice 0.692 138% 0.554 106%
9 Cobalt 0.938 186% 0.551 105%
10 Pimple 1.189 236% 0.571 109%
11 Joomla 1.426 283% 0.599 114%
12 Opulence IoC 1.786 355% 0.553 106%
13 Disco 2.215 440% 0.635 121%
14 Laravel 2.314 460% 0.544 104%
15 PHPixie DI 2.535 504% 0.565 108%
16 Auryn 4.078 811% 0.56 107%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.052 100% 0.524 100%
2 Symfony 5.213 103% 0.531 101%
3 Yaco 5.942 118% 0.563 107%
4 PHP-DI 6.016 119% 0.53 101%
5 Zend ServiceManager 6.151 122% 0.566 108%
6 Aura 6.269 124% 0.579 111%
7 Yii2 Container 6.665 132% 0.601 115%
8 Dice 6.943 137% 0.554 106%
9 Cobalt 9.35 185% 0.551 105%
10 Pimple 11.836 234% 0.571 109%
11 Joomla 14.241 282% 0.599 114%
12 Opulence IoC 17.765 352% 0.553 106%
13 Disco 21.973 435% 0.635 121%
14 Laravel 23.012 456% 0.544 104%
15 PHPixie DI 25.242 500% 0.565 108%
16 Auryn 40.515 802% 0.56 107%

Test Suite 6: "Warm" Fetching of the same big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.051 100% 0.622 100%
2 Symfony 0.053 104% 0.64 103%
3 Yaco 0.06 118% 0.67 108%
4 PHP-DI 0.061 120% 0.648 104%
5 Zend ServiceManager 0.062 122% 0.681 110%
6 Aura 0.064 125% 0.894 144%
7 Yii2 Container 0.067 131% 0.822 132%
8 Dice 0.07 137% 0.944 152%
9 Cobalt 0.094 184% 0.863 139%
10 Pimple 0.119 233% 0.686 110%
11 Joomla 0.142 278% 0.706 114%
12 Opulence IoC 0.179 351% 0.753 121%
13 Disco 0.224 439% 0.767 123%
14 Laravel 0.231 453% 0.718 116%
15 PHPixie DI 0.255 500% 0.663 107%
16 Auryn 0.407 798% 0.828 133%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.503 100% 0.622 100%
2 Symfony 0.524 104% 0.64 103%
3 Yaco 0.592 118% 0.67 108%
4 PHP-DI 0.599 119% 0.648 104%
5 Zend ServiceManager 0.612 122% 0.681 110%
6 Aura 0.625 124% 0.894 144%
7 Yii2 Container 0.663 132% 0.822 132%
8 Dice 0.691 137% 0.944 152%
9 Cobalt 0.931 185% 0.863 139%
10 Pimple 1.184 235% 0.686 110%
11 Joomla 1.424 283% 0.706 114%
12 Opulence IoC 1.788 355% 0.753 121%
13 Disco 2.246 447% 0.767 123%
14 Laravel 2.314 460% 0.718 116%
15 PHPixie DI 2.563 510% 0.663 107%
16 Auryn 4.095 814% 0.828 133%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.065 100% 0.622 100%
2 Symfony 5.255 104% 0.64 103%
3 Yaco 5.948 117% 0.67 108%
4 PHP-DI 6.022 119% 0.648 104%
5 Zend ServiceManager 6.136 121% 0.681 110%
6 Aura 6.273 124% 0.894 144%
7 Yii2 Container 6.673 132% 0.822 132%
8 Dice 6.936 137% 0.944 152%
9 Cobalt 9.33 184% 0.863 139%
10 Pimple 11.833 234% 0.686 110%
11 Joomla 14.229 281% 0.706 114%
12 Opulence IoC 17.756 351% 0.753 121%
13 Disco 22.309 440% 0.767 123%
14 Laravel 23.054 455% 0.718 116%
15 PHPixie DI 25.385 501% 0.663 107%
16 Auryn 40.726 804% 0.828 133%


Conclusion

Keep in mind that in a well-architected application you won't call your DI Container hundreds or even thousands of times because ideally there should be as few composition roots as possible (but there is a good chance of needing the container in other places of the application layer - e.g. in your middleware or bootstrap files). That's why most results are exaggerated - you probably won't see milliseconds of difference between the fastest and the slowest DIC in the real life.

To sum up, when choosing a container it only depends on your needs which one suits your project best: if you have a performance-critical application then you probably want to choose a compiled container. If maximum performance is not required, but you develop a big and complex system then I would recommend a dynamic container with autowiring capabilities. Otherwise you can go with simpler containers.