DI Container Benchmark

Author:Máté Kocsis (@kocsismate90)
Repository:https://github.com/kocsismate/php-di-container-benchmarks
Generated:2019-04-16 16:18:20

Table of Contents


Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

Name Version Compiled/Dynamic Autowiring
aura/di 3.4.0 dynamic supported
rdlowrey/auryn v1.4.2 dynamic supported
jshannon63/cobalt v1.2.1 dynamic supported
level-2/dice 4.0 dynamic supported
bitexpert/disco v0.10.0 compiled not supported
joomla/di 1.5.1 dynamic supported
illuminate/container v5.8.12 dynamic supported
opulence/ioc v1.1.7 dynamic supported
php-di/php-di 6.0.7 compiled supported
phpixie/di 3.2.1 dynamic not supported
pimple/pimple v3.2.3 dynamic not supported
symfony/dependency-injection v4.2.6 compiled supported
thecodingmachine/yaco v1.0.0 compiled not supported
yiisoft/yii2 2.0.17 dynamic supported
woohoolabs/zen 2.7.2 compiled supported
zendframework/zend-servicemanager 3.4.0 dynamic not supported

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.


Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (10 or 100 objects). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Suites: "Cold" ones (Test Suite 1-2) measure performance including autoloading and startup time of containers as well as autoloading time of the retrieved objects. "Semi-Warm" ones (Test Suite 3-4) measure performance excluding container autoloading time, but including startup time and autoloading time of the retrieved objects, while "Warm" ones (Test Suite 5-6) exclude autoloading and startup time equally. Time of compilation is always excluded from the results due to OPcache.

Each Test Suite contains three Test Cases which define the number of iterations the main task has to be repeated in order to simulate real world usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results are displayed in the final results.


Setup

The benchmark is run on AWS EC2, using a C5.large instance. The operating system on the host is Ubuntu 18.04. PHP 7.3 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.


Results

Test Suite 1: "Cold" Retrieval of a small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.038 100% 0.524 100%
2 Symfony 0.05 132% 0.531 101%
3 Yaco 0.067 176% 0.563 107%
4 PHP-DI 0.079 208% 0.53 101%
5 PHPixie DI 0.083 218% 0.565 108%
6 Zend ServiceManager 0.086 226% 0.566 108%
7 Joomla 0.089 234% 0.599 114%
8 Pimple 0.091 239% 0.571 109%
9 Dice 0.1 263% 0.554 106%
10 Cobalt 0.108 284% 0.551 105%
11 Laravel 0.111 292% 0.544 104%
12 Opulence IoC 0.12 316% 0.553 106%
13 Yii2 Container 0.131 345% 0.601 115%
14 Auryn 0.193 508% 0.56 107%
15 Aura 0.194 511% 0.579 111%
16 Disco 0.262 689% 0.635 121%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.043 100% 0.524 100%
2 Symfony 0.054 126% 0.531 101%
3 Yaco 0.072 167% 0.563 107%
4 PHP-DI 0.085 198% 0.53 101%
5 Zend ServiceManager 0.092 214% 0.566 108%
6 Dice 0.094 219% 0.554 106%
7 Pimple 0.101 235% 0.571 109%
8 Joomla 0.102 237% 0.599 114%
9 PHPixie DI 0.106 247% 0.565 108%
10 Cobalt 0.113 263% 0.551 105%
11 Laravel 0.131 305% 0.544 104%
12 Yii2 Container 0.135 314% 0.601 115%
13 Opulence IoC 0.137 319% 0.553 106%
14 Aura 0.2 465% 0.579 111%
15 Auryn 0.209 486% 0.56 107%
16 Disco 0.278 647% 0.635 121%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.091 100% 0.524 100%
2 Symfony 0.103 113% 0.531 101%
3 Yaco 0.127 140% 0.563 107%
4 PHP-DI 0.14 154% 0.53 101%
5 Zend ServiceManager 0.148 163% 0.566 108%
6 Dice 0.157 173% 0.554 106%
7 Yii2 Container 0.197 216% 0.601 115%
8 Cobalt 0.199 219% 0.551 105%
9 Pimple 0.211 232% 0.571 109%
10 Joomla 0.229 252% 0.599 114%
11 Aura 0.258 284% 0.579 111%
12 Opulence IoC 0.303 333% 0.553 106%
13 PHPixie DI 0.337 370% 0.565 108%
14 Laravel 0.34 374% 0.544 104%
15 Disco 0.478 525% 0.635 121%
16 Auryn 0.552 607% 0.56 107%

Test Suite 2: "Cold" Retrieval of a big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.148 100% 0.64 100%
2 Zen 0.231 156% 0.622 97%
3 Yaco 0.273 184% 0.67 105%
4 PHPixie DI 0.294 199% 0.663 104%
5 Joomla 0.309 209% 0.706 110%
6 PHP-DI 0.309 209% 0.648 101%
7 Zend ServiceManager 0.319 216% 0.681 106%
8 Pimple 0.342 231% 0.686 107%
9 Opulence IoC 0.505 341% 0.753 118%
10 Disco 0.527 356% 0.767 120%
11 Yii2 Container 0.531 359% 0.822 128%
12 Cobalt 0.533 360% 0.863 135%
13 Laravel 0.54 365% 0.718 112%
14 Dice 0.558 377% 0.944 147%
15 Aura 0.72 486% 0.894 140%
16 Auryn 0.84 568% 0.828 129%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.153 100% 0.64 100%
2 Zen 0.24 157% 0.622 97%
3 Yaco 0.277 181% 0.67 105%
4 PHP-DI 0.316 207% 0.648 101%
5 PHPixie DI 0.319 208% 0.663 104%
6 Zend ServiceManager 0.322 210% 0.681 106%
7 Joomla 0.324 212% 0.706 110%
8 Pimple 0.352 230% 0.686 107%
9 Opulence IoC 0.523 342% 0.753 118%
10 Cobalt 0.541 354% 0.863 135%
11 Yii2 Container 0.541 354% 0.822 128%
12 Disco 0.546 357% 0.767 120%
13 Dice 0.562 367% 0.944 147%
14 Laravel 0.565 369% 0.718 112%
15 Aura 0.724 473% 0.894 140%
16 Auryn 0.872 570% 0.828 129%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.201 100% 0.64 100%
2 Zen 0.286 142% 0.622 97%
3 Yaco 0.333 166% 0.67 105%
4 PHP-DI 0.37 184% 0.648 101%
5 Zend ServiceManager 0.381 190% 0.681 106%
6 Joomla 0.456 227% 0.706 110%
7 Pimple 0.462 230% 0.686 107%
8 PHPixie DI 0.547 272% 0.663 104%
9 Yii2 Container 0.599 298% 0.822 128%
10 Cobalt 0.626 311% 0.863 135%
11 Dice 0.629 313% 0.944 147%
12 Opulence IoC 0.692 344% 0.753 118%
13 Disco 0.752 374% 0.767 120%
14 Laravel 0.769 383% 0.718 112%
15 Aura 0.782 389% 0.894 140%
16 Auryn 1.229 611% 0.828 129%

Test Suite 3: "Semi-Warm" Instantiation of a small object graph (10 objects)

In this Test Suite, containers have to create an object graph of 10 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.031 100% 0.531 100%
2 Zen 0.033 106% 0.524 99%
3 PHPixie DI 0.071 229% 0.603 114%
4 Disco 0.073 235% 0.632 119%
5 Joomla 0.078 252% 0.673 127%
6 Cobalt 0.078 252% 0.559 105%
7 Zend ServiceManager 0.085 274% 0.604 114%
8 Dice 0.105 339% 0.545 103%
9 Pimple 0.126 406% 0.633 119%
10 Yii2 Container 0.157 506% 0.626 118%
11 Opulence IoC 0.167 539% 0.555 105%
12 Aura 0.175 565% 0.554 104%
13 Laravel 0.274 884% 0.548 103%
14 Auryn 0.353 1139% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.088 100% 0.531 100%
2 Zen 0.092 105% 0.524 99%
3 Disco 0.21 239% 0.632 119%
4 Cobalt 0.261 297% 0.559 105%
5 PHPixie DI 0.27 307% 0.603 114%
6 Joomla 0.281 319% 0.673 127%
7 Zend ServiceManager 0.379 431% 0.604 114%
8 Dice 0.438 498% 0.545 103%
9 Pimple 0.606 689% 0.633 119%
10 Yii2 Container 0.738 839% 0.626 118%
11 Aura 0.829 942% 0.554 104%
12 Opulence IoC 0.979 1113% 0.555 105%
13 Laravel 2.143 2435% 0.548 103%
14 Auryn 2.708 3077% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.664 100% 0.531 100%
2 Zen 0.674 102% 0.524 99%
3 Disco 1.589 239% 0.632 119%
4 Cobalt 2.102 317% 0.559 105%
5 PHPixie DI 2.271 342% 0.603 114%
6 Joomla 2.322 350% 0.673 127%
7 Zend ServiceManager 3.326 501% 0.604 114%
8 Dice 3.791 571% 0.545 103%
9 Pimple 5.412 815% 0.633 119%
10 Yii2 Container 6.567 989% 0.626 118%
11 Aura 7.372 1110% 0.554 104%
12 Opulence IoC 9.093 1369% 0.555 105%
13 Laravel 20.775 3129% 0.548 103%
14 Auryn 26.19 3944% 0.546 103%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 4: "Semi-Warm" Instantiation of a big object graph (100 objects)

In this Test Suite, containers have to create an object graph of 100 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.169 100% 0.636 100%
2 Zen 0.273 162% 0.622 98%
3 Disco 0.429 254% 0.728 115%
4 PHPixie DI 0.451 267% 0.703 110%
5 Joomla 0.498 295% 0.771 121%
6 Zend ServiceManager 0.579 343% 0.708 111%
7 Cobalt 0.663 392% 0.9 141%
8 Pimple 0.808 478% 0.679 107%
9 Dice 0.842 498% 0.856 135%
10 Yii2 Container 1.083 641% 0.814 128%
11 Opulence IoC 1.299 769% 0.752 118%
12 Aura 1.311 776% 0.869 137%
13 Laravel 2.375 1405% 0.729 115%
14 Auryn 3.253 1925% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.653 100% 0.636 100%
2 Zen 0.774 119% 0.622 98%
3 Disco 1.716 263% 0.728 115%
4 PHPixie DI 2.336 358% 0.703 110%
5 Cobalt 2.476 379% 0.9 141%
6 Joomla 2.67 409% 0.771 121%
7 Zend ServiceManager 3.477 532% 0.708 111%
8 Dice 4.304 659% 0.856 135%
9 Pimple 5.666 868% 0.679 107%
10 Yii2 Container 6.959 1066% 0.814 128%
11 Aura 7.942 1216% 0.869 137%
12 Opulence IoC 9.833 1506% 0.752 118%
13 Laravel 20.839 3191% 0.729 115%
14 Auryn 27.889 4271% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 5.503 100% 0.636 100%
2 Zen 5.799 105% 0.622 98%
3 Disco 14.446 263% 0.728 115%
4 Cobalt 20.524 373% 0.9 141%
5 PHPixie DI 21.071 383% 0.703 110%
6 Joomla 24.333 442% 0.771 121%
7 Zend ServiceManager 32.382 588% 0.708 111%
8 Dice 38.844 706% 0.856 135%
9 Pimple 53.801 978% 0.679 107%
10 Yii2 Container 65.569 1192% 0.814 128%
11 Aura 74.209 1349% 0.869 137%
12 Opulence IoC 95.036 1727% 0.752 118%
13 Laravel 205.498 3734% 0.729 115%
14 Auryn 273.898 4977% 0.814 128%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 5: "Warm" Fetching of the same small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.055 100% 0.531 100%
2 Zen 0.058 105% 0.524 99%
3 Yaco 0.062 113% 0.563 106%
4 PHP-DI 0.062 113% 0.53 100%
5 Zend ServiceManager 0.063 115% 0.566 107%
6 Aura 0.066 120% 0.579 109%
7 Yii2 Container 0.066 120% 0.601 113%
8 Dice 0.072 131% 0.554 104%
9 Cobalt 0.096 175% 0.551 104%
10 Pimple 0.122 222% 0.571 108%
11 Joomla 0.143 260% 0.599 113%
12 Opulence IoC 0.187 340% 0.553 104%
13 Disco 0.223 405% 0.635 120%
14 Laravel 0.232 422% 0.544 102%
15 PHPixie DI 0.255 464% 0.565 106%
16 Auryn 0.383 696% 0.56 106%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.544 100% 0.531 100%
2 Zen 0.574 106% 0.524 99%
3 Yaco 0.61 112% 0.563 106%
4 PHP-DI 0.611 112% 0.53 100%
5 Zend ServiceManager 0.624 115% 0.566 107%
6 Aura 0.652 120% 0.579 109%
7 Yii2 Container 0.656 121% 0.601 113%
8 Dice 0.709 130% 0.554 104%
9 Cobalt 0.954 175% 0.551 104%
10 Pimple 1.222 225% 0.571 108%
11 Joomla 1.419 261% 0.599 113%
12 Opulence IoC 1.86 342% 0.553 104%
13 Disco 2.229 410% 0.635 120%
14 Laravel 2.326 428% 0.544 102%
15 PHPixie DI 2.546 468% 0.565 106%
16 Auryn 3.83 704% 0.56 106%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 5.45 100% 0.531 100%
2 Zen 5.747 105% 0.524 99%
3 Yaco 6.12 112% 0.563 106%
4 PHP-DI 6.129 112% 0.53 100%
5 Zend ServiceManager 6.258 115% 0.566 107%
6 Aura 6.542 120% 0.579 109%
7 Yii2 Container 6.582 121% 0.601 113%
8 Dice 7.101 130% 0.554 104%
9 Cobalt 9.56 175% 0.551 104%
10 Pimple 12.115 222% 0.571 108%
11 Joomla 14.231 261% 0.599 113%
12 Opulence IoC 18.543 340% 0.553 104%
13 Disco 22.217 408% 0.635 120%
14 Laravel 23.183 425% 0.544 102%
15 PHPixie DI 25.402 466% 0.565 106%
16 Auryn 38.221 701% 0.56 106%

Test Suite 6: "Warm" Fetching of the same big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.054 100% 0.622 100%
2 Symfony 0.055 102% 0.64 103%
3 PHP-DI 0.062 115% 0.648 104%
4 Yaco 0.063 117% 0.67 108%
5 Zend ServiceManager 0.065 120% 0.681 110%
6 Aura 0.066 122% 0.894 144%
7 Yii2 Container 0.067 124% 0.822 132%
8 Dice 0.072 133% 0.944 152%
9 Cobalt 0.096 178% 0.863 139%
10 Pimple 0.122 226% 0.686 110%
11 Joomla 0.151 280% 0.706 114%
12 Opulence IoC 0.188 348% 0.753 121%
13 Disco 0.226 419% 0.767 123%
14 Laravel 0.233 431% 0.718 116%
15 PHPixie DI 0.257 476% 0.663 107%
16 Auryn 0.385 713% 0.828 133%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.53 100% 0.622 100%
2 Symfony 0.545 103% 0.64 103%
3 PHP-DI 0.612 115% 0.648 104%
4 Yaco 0.62 117% 0.67 108%
5 Zend ServiceManager 0.64 121% 0.681 110%
6 Aura 0.653 123% 0.894 144%
7 Yii2 Container 0.656 124% 0.822 132%
8 Dice 0.709 134% 0.944 152%
9 Cobalt 0.953 180% 0.863 139%
10 Pimple 1.211 228% 0.686 110%
11 Joomla 1.512 285% 0.706 114%
12 Opulence IoC 1.879 355% 0.753 121%
13 Disco 2.267 428% 0.767 123%
14 Laravel 2.327 439% 0.718 116%
15 PHPixie DI 2.571 485% 0.663 107%
16 Auryn 3.848 726% 0.828 133%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.32 100% 0.622 100%
2 Symfony 5.449 102% 0.64 103%
3 PHP-DI 6.131 115% 0.648 104%
4 Yaco 6.222 117% 0.67 108%
5 Zend ServiceManager 6.414 121% 0.681 110%
6 Aura 6.544 123% 0.894 144%
7 Yii2 Container 6.585 124% 0.822 132%
8 Dice 7.102 133% 0.944 152%
9 Cobalt 9.554 180% 0.863 139%
10 Pimple 12.119 228% 0.686 110%
11 Joomla 15.081 283% 0.706 114%
12 Opulence IoC 18.7 352% 0.753 121%
13 Disco 22.547 424% 0.767 123%
14 Laravel 23.195 436% 0.718 116%
15 PHPixie DI 25.645 482% 0.663 107%
16 Auryn 38.365 721% 0.828 133%


Conclusion

Keep in mind that in a well-architected application you won't call your DI Container hundreds or even thousands of times because ideally there should be as few composition roots as possible (but there is a good chance of needing the container in other places of the application layer - e.g. in your middleware or bootstrap files). That's why most results are exaggerated - you probably won't see milliseconds of difference between the fastest and the slowest DIC in the real life.

To sum up, when choosing a container it only depends on your needs which one suits your project best: if you have a performance-critical application then you probably want to choose a compiled container. If maximum performance is not required, but you develop a big and complex system then I would recommend a dynamic container with autowiring capabilities. Otherwise you can go with simpler containers.