DI Container Benchmark

Author:Máté Kocsis (@kocsismate90)
Repository:https://github.com/kocsismate/php-di-container-benchmarks
Generated:2019-07-08 17:49:21

Table of Contents


Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

Name Version Compiled/Dynamic Autowiring
aura/di 3.4.0 dynamic supported
rdlowrey/auryn v1.4.2 dynamic supported
jshannon63/cobalt v1.2.1 dynamic supported
level-2/dice 4.0.1 dynamic supported
bitexpert/disco v0.10.0 compiled not supported
joomla/di 1.5.1 dynamic supported
illuminate/container v5.8.27 dynamic supported
opulence/ioc v1.1.7 dynamic supported
php-di/php-di 6.0.9 compiled supported
phpixie/di 3.2.1 dynamic not supported
pimple/pimple v3.2.3 dynamic not supported
symfony/dependency-injection v4.3.2 compiled supported
thecodingmachine/yaco v1.0.0 compiled not supported
yiisoft/yii2 2.0.22 dynamic supported
woohoolabs/zen 2.7.2 compiled supported
zendframework/zend-servicemanager 3.4.0 dynamic not supported

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.


Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (10 or 100 objects). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Suites: "Cold" ones (Test Suite 1-2) measure performance including autoloading and startup time of containers as well as autoloading time of the retrieved objects. "Semi-Warm" ones (Test Suite 3-4) measure performance excluding container autoloading time, but including startup time and autoloading time of the retrieved objects, while "Warm" ones (Test Suite 5-6) exclude autoloading and startup time equally. Time of compilation is always excluded from the results due to OPcache.

Each Test Suite contains three Test Cases which define the number of iterations the main task has to be repeated in order to simulate real world usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results are displayed in the final results.


Setup

The benchmark is run on AWS EC2, using a C5.large instance. The operating system on the host is Ubuntu 18.04. PHP 7.3 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.


Results

Test Suite 1: "Cold" Retrieval of a small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.039 100% 0.524 100%
2 Symfony 0.048 123% 0.531 101%
3 Yaco 0.067 172% 0.563 107%
4 PHP-DI 0.078 200% 0.53 101%
5 PHPixie DI 0.082 210% 0.565 108%
6 Zend ServiceManager 0.083 213% 0.566 108%
7 Dice 0.087 223% 0.554 106%
8 Joomla 0.088 226% 0.599 114%
9 Pimple 0.089 228% 0.571 109%
10 Cobalt 0.105 269% 0.551 105%
11 Laravel 0.111 285% 0.544 104%
12 Opulence IoC 0.122 313% 0.553 106%
13 Yii2 Container 0.124 318% 0.601 115%
14 Auryn 0.175 449% 0.56 107%
15 Aura 0.192 492% 0.579 111%
16 Disco 0.256 656% 0.635 121%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.043 100% 0.524 100%
2 Symfony 0.054 126% 0.531 101%
3 Yaco 0.072 167% 0.563 107%
4 PHP-DI 0.083 193% 0.53 101%
5 Zend ServiceManager 0.089 207% 0.566 108%
6 Dice 0.094 219% 0.554 106%
7 Pimple 0.1 233% 0.571 109%
8 Joomla 0.103 240% 0.599 114%
9 PHPixie DI 0.105 244% 0.565 108%
10 Cobalt 0.113 263% 0.551 105%
11 Yii2 Container 0.128 298% 0.601 115%
12 Laravel 0.132 307% 0.544 104%
13 Opulence IoC 0.137 319% 0.553 106%
14 Aura 0.198 460% 0.579 111%
15 Auryn 0.21 488% 0.56 107%
16 Disco 0.276 642% 0.635 121%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.091 100% 0.524 100%
2 Symfony 0.103 113% 0.531 101%
3 Yaco 0.128 141% 0.563 107%
4 PHP-DI 0.139 153% 0.53 101%
5 Zend ServiceManager 0.145 159% 0.566 108%
6 Dice 0.158 174% 0.554 106%
7 Yii2 Container 0.19 209% 0.601 115%
8 Cobalt 0.197 216% 0.551 105%
9 Pimple 0.207 227% 0.571 109%
10 Joomla 0.232 255% 0.599 114%
11 Aura 0.256 281% 0.579 111%
12 Opulence IoC 0.299 329% 0.553 106%
13 PHPixie DI 0.331 364% 0.565 108%
14 Laravel 0.339 373% 0.544 104%
15 Disco 0.479 526% 0.635 121%
16 Auryn 0.553 608% 0.56 107%

Test Suite 2: "Cold" Retrieval of a big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.145 100% 0.64 100%
2 Zen 0.23 159% 0.622 97%
3 Yaco 0.272 188% 0.67 105%
4 PHPixie DI 0.294 203% 0.663 104%
5 Joomla 0.31 214% 0.706 110%
6 PHP-DI 0.312 215% 0.648 101%
7 Zend ServiceManager 0.318 219% 0.681 106%
8 Pimple 0.344 237% 0.686 107%
9 Opulence IoC 0.508 350% 0.753 118%
10 Disco 0.519 358% 0.767 120%
11 Cobalt 0.532 367% 0.863 135%
12 Yii2 Container 0.534 368% 0.822 128%
13 Laravel 0.54 372% 0.718 112%
14 Dice 0.552 381% 0.944 147%
15 Aura 0.71 490% 0.894 140%
16 Auryn 0.841 580% 0.828 129%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.152 100% 0.64 100%
2 Zen 0.234 154% 0.622 97%
3 Yaco 0.281 185% 0.67 105%
4 PHPixie DI 0.314 207% 0.663 104%
5 PHP-DI 0.319 210% 0.648 101%
6 Joomla 0.323 213% 0.706 110%
7 Zend ServiceManager 0.325 214% 0.681 106%
8 Pimple 0.354 233% 0.686 107%
9 Opulence IoC 0.528 347% 0.753 118%
10 Yii2 Container 0.54 355% 0.822 128%
11 Cobalt 0.542 357% 0.863 135%
12 Disco 0.543 357% 0.767 120%
13 Dice 0.559 368% 0.944 147%
14 Laravel 0.562 370% 0.718 112%
15 Aura 0.72 474% 0.894 140%
16 Auryn 0.88 579% 0.828 129%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.2 100% 0.64 100%
2 Zen 0.281 141% 0.622 97%
3 Yaco 0.332 166% 0.67 105%
4 PHP-DI 0.376 188% 0.648 101%
5 Zend ServiceManager 0.378 189% 0.681 106%
6 Joomla 0.456 228% 0.706 110%
7 Pimple 0.461 231% 0.686 107%
8 PHPixie DI 0.546 273% 0.663 104%
9 Yii2 Container 0.596 298% 0.822 128%
10 Dice 0.623 312% 0.944 147%
11 Cobalt 0.63 315% 0.863 135%
12 Opulence IoC 0.688 344% 0.753 118%
13 Disco 0.743 372% 0.767 120%
14 Laravel 0.771 386% 0.718 112%
15 Aura 0.778 389% 0.894 140%
16 Auryn 1.227 614% 0.828 129%

Test Suite 3: "Semi-Warm" Instantiation of a small object graph (10 objects)

In this Test Suite, containers have to create an object graph of 10 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.029 100% 0.531 100%
2 Zen 0.033 114% 0.524 99%
3 PHPixie DI 0.071 245% 0.603 114%
4 Disco 0.071 245% 0.632 119%
5 Joomla 0.078 269% 0.673 127%
6 Cobalt 0.078 269% 0.559 105%
7 Zend ServiceManager 0.085 293% 0.604 114%
8 Dice 0.105 362% 0.545 103%
9 Pimple 0.127 438% 0.633 119%
10 Yii2 Container 0.154 531% 0.626 118%
11 Opulence IoC 0.167 576% 0.555 105%
12 Aura 0.174 600% 0.554 104%
13 Laravel 0.273 941% 0.548 103%
14 Auryn 0.352 1214% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.085 100% 0.531 100%
2 Zen 0.09 106% 0.524 99%
3 Disco 0.207 244% 0.632 119%
4 Cobalt 0.261 307% 0.559 105%
5 PHPixie DI 0.269 316% 0.603 114%
6 Joomla 0.283 333% 0.673 127%
7 Zend ServiceManager 0.384 452% 0.604 114%
8 Dice 0.442 520% 0.545 103%
9 Pimple 0.607 714% 0.633 119%
10 Yii2 Container 0.728 856% 0.626 118%
11 Aura 0.824 969% 0.554 104%
12 Opulence IoC 0.971 1142% 0.555 105%
13 Laravel 2.118 2492% 0.548 103%
14 Auryn 2.733 3215% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.643 100% 0.531 100%
2 Zen 0.658 102% 0.524 99%
3 Disco 1.561 243% 0.632 119%
4 Cobalt 2.095 326% 0.559 105%
5 PHPixie DI 2.257 351% 0.603 114%
6 Joomla 2.345 365% 0.673 127%
7 Zend ServiceManager 3.399 529% 0.604 114%
8 Dice 3.812 593% 0.545 103%
9 Pimple 5.41 841% 0.633 119%
10 Yii2 Container 6.498 1011% 0.626 118%
11 Aura 7.293 1134% 0.554 104%
12 Opulence IoC 9.022 1403% 0.555 105%
13 Laravel 20.563 3198% 0.548 103%
14 Auryn 26.566 4132% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 4: "Semi-Warm" Instantiation of a big object graph (100 objects)

In this Test Suite, containers have to create an object graph of 100 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.166 100% 0.636 100%
2 Zen 0.269 162% 0.622 98%
3 Disco 0.43 259% 0.728 115%
4 PHPixie DI 0.448 270% 0.703 110%
5 Joomla 0.499 301% 0.771 121%
6 Zend ServiceManager 0.58 349% 0.708 111%
7 Cobalt 0.657 396% 0.9 141%
8 Pimple 0.803 484% 0.679 107%
9 Dice 0.84 506% 0.856 135%
10 Yii2 Container 1.09 657% 0.814 128%
11 Aura 1.291 778% 0.869 137%
12 Opulence IoC 1.308 788% 0.752 118%
13 Laravel 2.364 1424% 0.729 115%
14 Auryn 3.263 1966% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.643 100% 0.636 100%
2 Zen 0.761 118% 0.622 98%
3 Disco 1.733 270% 0.728 115%
4 PHPixie DI 2.301 358% 0.703 110%
5 Cobalt 2.449 381% 0.9 141%
6 Joomla 2.687 418% 0.771 121%
7 Zend ServiceManager 3.509 546% 0.708 111%
8 Dice 4.325 673% 0.856 135%
9 Pimple 5.611 873% 0.679 107%
10 Yii2 Container 6.92 1076% 0.814 128%
11 Aura 7.84 1219% 0.869 137%
12 Opulence IoC 9.871 1535% 0.752 118%
13 Laravel 20.733 3224% 0.729 115%
14 Auryn 28.08 4367% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 5.43 100% 0.636 100%
2 Zen 5.666 104% 0.622 98%
3 Disco 14.612 269% 0.728 115%
4 Cobalt 20.336 375% 0.9 141%
5 PHPixie DI 20.819 383% 0.703 110%
6 Joomla 24.507 451% 0.771 121%
7 Zend ServiceManager 32.762 603% 0.708 111%
8 Dice 39.093 720% 0.856 135%
9 Pimple 53.1 978% 0.679 107%
10 Yii2 Container 65.182 1200% 0.814 128%
11 Aura 73.376 1351% 0.869 137%
12 Opulence IoC 95.339 1756% 0.752 118%
13 Laravel 204.393 3764% 0.729 115%
14 Auryn 275.465 5073% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 5: "Warm" Fetching of the same small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.054 100% 0.524 100%
2 Symfony 0.055 102% 0.531 101%
3 Yaco 0.061 113% 0.563 107%
4 Zend ServiceManager 0.062 115% 0.566 108%
5 PHP-DI 0.063 117% 0.53 101%
6 Aura 0.065 120% 0.579 111%
7 Yii2 Container 0.067 124% 0.601 115%
8 Dice 0.071 131% 0.554 106%
9 Cobalt 0.1 185% 0.551 105%
10 Pimple 0.12 222% 0.571 109%
11 Joomla 0.145 269% 0.599 114%
12 Opulence IoC 0.181 335% 0.553 106%
13 Disco 0.225 417% 0.635 121%
14 Laravel 0.23 426% 0.544 104%
15 PHPixie DI 0.252 467% 0.565 108%
16 Auryn 0.384 711% 0.56 107%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.529 100% 0.524 100%
2 Symfony 0.546 103% 0.531 101%
3 Yaco 0.602 114% 0.563 107%
4 Zend ServiceManager 0.616 116% 0.566 108%
5 PHP-DI 0.623 118% 0.53 101%
6 Aura 0.64 121% 0.579 111%
7 Yii2 Container 0.658 124% 0.601 115%
8 Dice 0.708 134% 0.554 106%
9 Cobalt 0.992 188% 0.551 105%
10 Pimple 1.193 226% 0.571 109%
11 Joomla 1.448 274% 0.599 114%
12 Opulence IoC 1.812 343% 0.553 106%
13 Disco 2.264 428% 0.635 121%
14 Laravel 2.302 435% 0.544 104%
15 PHPixie DI 2.534 479% 0.565 108%
16 Auryn 3.839 726% 0.56 107%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.304 100% 0.524 100%
2 Symfony 5.492 104% 0.531 101%
3 Yaco 6.035 114% 0.563 107%
4 Zend ServiceManager 6.161 116% 0.566 108%
5 PHP-DI 6.246 118% 0.53 101%
6 Aura 6.42 121% 0.579 111%
7 Yii2 Container 6.593 124% 0.601 115%
8 Dice 7.097 134% 0.554 106%
9 Cobalt 9.959 188% 0.551 105%
10 Pimple 11.933 225% 0.571 109%
11 Joomla 14.492 273% 0.599 114%
12 Opulence IoC 18.042 340% 0.553 106%
13 Disco 22.479 424% 0.635 121%
14 Laravel 22.977 433% 0.544 104%
15 PHPixie DI 25.325 477% 0.565 108%
16 Auryn 38.348 723% 0.56 107%

Test Suite 6: "Warm" Fetching of the same big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.054 100% 0.622 100%
2 Symfony 0.055 102% 0.64 103%
3 Yaco 0.061 113% 0.67 108%
4 Zend ServiceManager 0.062 115% 0.681 110%
5 PHP-DI 0.063 117% 0.648 104%
6 Aura 0.066 122% 0.894 144%
7 Yii2 Container 0.067 124% 0.822 132%
8 Dice 0.072 133% 0.944 152%
9 Cobalt 0.098 181% 0.863 139%
10 Pimple 0.121 224% 0.686 110%
11 Joomla 0.146 270% 0.706 114%
12 Opulence IoC 0.183 339% 0.753 121%
13 Disco 0.224 415% 0.767 123%
14 Laravel 0.23 426% 0.718 116%
15 PHPixie DI 0.254 470% 0.663 107%
16 Auryn 0.386 715% 0.828 133%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.535 100% 0.622 100%
2 Symfony 0.548 102% 0.64 103%
3 Yaco 0.602 113% 0.67 108%
4 Zend ServiceManager 0.614 115% 0.681 110%
5 PHP-DI 0.623 116% 0.648 104%
6 Aura 0.649 121% 0.894 144%
7 Yii2 Container 0.657 123% 0.822 132%
8 Dice 0.708 132% 0.944 152%
9 Cobalt 0.97 181% 0.863 139%
10 Pimple 1.2 224% 0.686 110%
11 Joomla 1.457 272% 0.706 114%
12 Opulence IoC 1.828 342% 0.753 121%
13 Disco 2.247 420% 0.767 123%
14 Laravel 2.309 432% 0.718 116%
15 PHPixie DI 2.546 476% 0.663 107%
16 Auryn 3.863 722% 0.828 133%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.361 100% 0.622 100%
2 Symfony 5.49 102% 0.64 103%
3 Yaco 6.044 113% 0.67 108%
4 Zend ServiceManager 6.168 115% 0.681 110%
5 PHP-DI 6.252 117% 0.648 104%
6 Aura 6.504 121% 0.894 144%
7 Yii2 Container 6.567 122% 0.822 132%
8 Dice 7.095 132% 0.944 152%
9 Cobalt 9.728 181% 0.863 139%
10 Pimple 12.009 224% 0.686 110%
11 Joomla 14.493 270% 0.706 114%
12 Opulence IoC 18.231 340% 0.753 121%
13 Disco 22.282 416% 0.767 123%
14 Laravel 22.999 429% 0.718 116%
15 PHPixie DI 25.376 473% 0.663 107%
16 Auryn 38.547 719% 0.828 133%


Conclusion

Keep in mind that in a well-architected application you won't call your DI Container hundreds or even thousands of times because ideally there should be as few composition roots as possible (but there is a good chance of needing the container in other places of the application layer - e.g. in your middleware or bootstrap files). That's why most results are exaggerated - you probably won't see milliseconds of difference between the fastest and the slowest DIC in the real life.

To sum up, when choosing a container it only depends on your needs which one suits your project best: if you have a performance-critical application then you probably want to choose a compiled container. If maximum performance is not required, but you develop a big and complex system then I would recommend a dynamic container with autowiring capabilities. Otherwise you can go with simpler containers.