DI Container Benchmark

Author:Máté Kocsis (@kocsismate90)
Repository:https://github.com/kocsismate/php-di-container-benchmarks
Generated:2019-08-14 13:12:26

Table of Contents


Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

Name Version Compiled/Dynamic Autowiring
aura/di 3.4.0 dynamic supported
rdlowrey/auryn v1.4.2 dynamic supported
jshannon63/cobalt v1.2.1 dynamic supported
level-2/dice 4.0.1 dynamic supported
bitexpert/disco v0.10.0 compiled not supported
joomla/di 1.5.1 dynamic supported
illuminate/container v5.8.32 dynamic supported
opulence/ioc v1.1.7 dynamic supported
php-di/php-di 6.0.9 compiled supported
phpixie/di 3.2.1 dynamic not supported
pimple/pimple v3.2.3 dynamic not supported
symfony/dependency-injection v4.3.3 compiled supported
thecodingmachine/yaco v1.0.0 compiled not supported
yiisoft/yii2 2.0.25 dynamic supported
woohoolabs/zen dev-master compiled supported
zendframework/zend-servicemanager 3.4.0 dynamic not supported

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.


Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (10 or 100 objects). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Suites: "Cold" ones (Test Suite 1-2) measure performance including autoloading and startup time of containers as well as autoloading time of the retrieved objects. "Semi-Warm" ones (Test Suite 3-4) measure performance excluding container autoloading time, but including startup time and autoloading time of the retrieved objects, while "Warm" ones (Test Suite 5-6) exclude autoloading and startup time equally. Time of compilation is always excluded from the results due to OPcache.

Each Test Suite contains three Test Cases which define the number of iterations the main task has to be repeated in order to simulate real world usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results are displayed in the final results.


Setup

The benchmark is run on AWS EC2, using a C5.large instance. The operating system on the host is Ubuntu 18.04. PHP 7.3 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.


Results

Test Suite 1: "Cold" Retrieval of a small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.032 100% 0.351 100%
2 Symfony 0.048 150% 0.42 120%
3 Yaco 0.059 184% 0.393 112%
4 Zend ServiceManager 0.07 219% 0.462 132%
5 PHP-DI 0.077 241% 0.354 101%
6 PHPixie DI 0.077 241% 0.395 112%
7 Dice 0.082 256% 0.382 109%
8 Pimple 0.082 256% 0.399 114%
9 Joomla 0.083 259% 0.43 122%
10 Cobalt 0.097 303% 0.44 125%
11 Laravel 0.107 334% 0.434 123%
12 Opulence IoC 0.114 356% 0.442 126%
13 Yii2 Container 0.115 359% 0.487 138%
14 Auryn 0.163 509% 0.382 109%
15 Aura 0.181 566% 0.46 131%
16 Disco 1.528 4775% 1.195 340%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.036 100% 0.351 100%
2 Symfony 0.049 136% 0.42 120%
3 Yaco 0.066 183% 0.393 112%
4 PHP-DI 0.074 206% 0.354 101%
5 Zend ServiceManager 0.083 231% 0.462 132%
6 Dice 0.086 239% 0.382 109%
7 Pimple 0.091 253% 0.399 114%
8 PHPixie DI 0.093 258% 0.395 112%
9 Joomla 0.099 275% 0.43 122%
10 Cobalt 0.105 292% 0.44 125%
11 Yii2 Container 0.121 336% 0.487 138%
12 Laravel 0.127 353% 0.434 123%
13 Opulence IoC 0.128 356% 0.442 126%
14 Aura 0.175 486% 0.46 131%
15 Auryn 0.2 556% 0.382 109%
16 Disco 0.252 700% 0.445 127%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.079 100% 0.351 100%
2 Symfony 0.098 124% 0.42 120%
3 Yaco 0.117 148% 0.393 112%
4 PHP-DI 0.132 167% 0.354 101%
5 Zend ServiceManager 0.134 170% 0.462 132%
6 Dice 0.146 185% 0.382 109%
7 Yii2 Container 0.173 219% 0.487 138%
8 Cobalt 0.184 233% 0.44 125%
9 Pimple 0.2 253% 0.399 114%
10 Joomla 0.225 285% 0.43 122%
11 Aura 0.237 300% 0.46 131%
12 Opulence IoC 0.311 394% 0.442 126%
13 PHPixie DI 0.322 408% 0.395 112%
14 Laravel 0.339 429% 0.434 123%
15 Disco 0.447 566% 0.445 127%
16 Auryn 0.533 675% 0.382 109%

Test Suite 2: "Cold" Retrieval of a big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 10, 100 and 1000 times. Autoloading and startup time of the containers are included in the measurements as well as autoloading time of the retrieved objects. That's why this Test Suite simulates production usage very well.

10 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.128 100% 0.441 100%
2 Zen 0.193 151% 0.359 82%
3 Yaco 0.231 180% 0.411 93%
4 Joomla 0.257 201% 0.448 102%
5 PHPixie DI 0.259 202% 0.403 91%
6 PHP-DI 0.262 205% 0.446 101%
7 Zend ServiceManager 0.268 209% 0.485 110%
8 Pimple 0.299 234% 0.425 96%
9 Opulence IoC 0.453 354% 0.549 125%
10 Cobalt 0.471 368% 0.662 150%
11 Disco 0.476 372% 0.488 111%
12 Yii2 Container 0.48 375% 0.584 133%
13 Laravel 0.496 388% 0.514 117%
14 Dice 0.524 409% 0.705 160%
15 Aura 0.653 510% 0.651 148%
16 Auryn 0.779 609% 0.607 138%

100 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.119 100% 0.441 100%
2 Zen 0.193 162% 0.359 82%
3 Yaco 0.235 197% 0.411 93%
4 Joomla 0.27 227% 0.448 102%
5 PHPixie DI 0.278 234% 0.403 91%
6 Zend ServiceManager 0.289 243% 0.485 110%
7 PHP-DI 0.294 247% 0.446 101%
8 Pimple 0.316 266% 0.425 96%
9 Cobalt 0.469 394% 0.662 150%
10 Opulence IoC 0.485 408% 0.549 125%
11 Yii2 Container 0.489 411% 0.584 133%
12 Dice 0.503 423% 0.705 160%
13 Disco 0.515 433% 0.488 111%
14 Laravel 0.52 437% 0.514 117%
15 Aura 0.638 536% 0.651 148%
16 Auryn 0.818 687% 0.607 138%

1000 iterations, autoload + startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.18 100% 0.441 100%
2 Zen 0.251 139% 0.359 82%
3 Yaco 0.295 164% 0.411 93%
4 PHP-DI 0.348 193% 0.446 101%
5 Zend ServiceManager 0.351 195% 0.485 110%
6 Joomla 0.429 238% 0.448 102%
7 Pimple 0.435 242% 0.425 96%
8 PHPixie DI 0.523 291% 0.403 91%
9 Yii2 Container 0.564 313% 0.584 133%
10 Cobalt 0.585 325% 0.662 150%
11 Dice 0.594 330% 0.705 160%
12 Disco 0.654 363% 0.488 111%
13 Opulence IoC 0.672 373% 0.549 125%
14 Laravel 0.72 400% 0.514 117%
15 Aura 0.722 401% 0.651 148%
16 Auryn 1.169 649% 0.607 138%

Test Suite 3: "Semi-Warm" Instantiation of a small object graph (10 objects)

In this Test Suite, containers have to create an object graph of 10 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.027 100% 0.42 100%
2 Zen 0.031 115% 0.351 84%
3 PHPixie DI 0.067 248% 0.437 104%
4 Cobalt 0.074 274% 0.448 107%
5 Joomla 0.078 289% 0.508 121%
6 Disco 0.078 289% 0.956 228%
7 Zend ServiceManager 0.084 311% 0.505 120%
8 Dice 0.099 367% 0.372 89%
9 Pimple 0.123 456% 0.466 111%
10 Yii2 Container 0.148 548% 0.519 124%
11 Aura 0.153 567% 0.436 104%
12 Opulence IoC 0.157 581% 0.384 91%
13 Laravel 0.252 933% 0.438 104%
14 Auryn 0.335 1241% 0.368 88%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.081 100% 0.42 100%
2 Zen 0.085 105% 0.351 84%
3 Disco 0.201 248% 0.444 106%
4 Cobalt 0.257 317% 0.448 107%
5 PHPixie DI 0.262 323% 0.437 104%
6 Joomla 0.286 353% 0.508 121%
7 Zend ServiceManager 0.375 463% 0.505 120%
8 Dice 0.432 533% 0.372 89%
9 Pimple 0.6 741% 0.466 111%
10 Yii2 Container 0.699 863% 0.519 124%
11 Aura 0.745 920% 0.436 104%
12 Opulence IoC 0.919 1135% 0.384 91%
13 Laravel 1.933 2386% 0.438 104%
14 Auryn 2.61 3222% 0.368 88%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.616 100% 0.42 100%
2 Zen 0.621 101% 0.351 84%
3 Disco 1.492 242% 0.444 106%
4 Cobalt 2.095 340% 0.448 107%
5 PHPixie DI 2.195 356% 0.437 104%
6 Joomla 2.395 389% 0.508 121%
7 Zend ServiceManager 3.299 536% 0.505 120%
8 Dice 3.77 612% 0.372 89%
9 Pimple 5.365 871% 0.466 111%
10 Yii2 Container 6.204 1007% 0.519 124%
11 Aura 6.61 1073% 0.436 104%
12 Opulence IoC 8.496 1379% 0.384 91%
13 Laravel 18.697 3035% 0.438 104%
14 Auryn 24.956 4051% 0.368 88%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 4: "Semi-Warm" Instantiation of a big object graph (100 objects)

In this Test Suite, containers have to create an object graph of 100 objects (defined as Prototypes) 10, 100 and 1000 times. Container autoloading time is excluded while startup time of the container and autoloading time of the retrieved objects are included in the measurements.

10 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.152 100% 0.436 100%
2 Zen 0.241 159% 0.359 82%
3 Disco 0.402 264% 0.456 104%
4 PHPixie DI 0.419 276% 0.445 102%
5 Joomla 0.479 315% 0.516 118%
6 Zend ServiceManager 0.554 364% 0.515 118%
7 Cobalt 0.636 418% 0.701 161%
8 Pimple 0.778 512% 0.466 107%
9 Dice 0.801 527% 0.61 140%
10 Yii2 Container 1.018 670% 0.576 132%
11 Aura 1.171 770% 0.626 144%
12 Opulence IoC 1.2 789% 0.484 111%
13 Laravel 2.135 1405% 0.525 120%
14 Auryn 3.098 2038% 0.543 124%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

100 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.612 100% 0.436 100%
2 Zen 0.703 115% 0.359 82%
3 Disco 1.655 270% 0.456 104%
4 PHPixie DI 2.266 370% 0.445 102%
5 Cobalt 2.442 399% 0.701 161%
6 Joomla 2.68 438% 0.516 118%
7 Zend ServiceManager 3.456 565% 0.515 118%
8 Dice 4.206 687% 0.61 140%
9 Pimple 5.508 900% 0.466 107%
10 Yii2 Container 6.55 1070% 0.576 132%
11 Aura 7.057 1153% 0.626 144%
12 Opulence IoC 9.078 1483% 0.484 111%
13 Laravel 18.611 3041% 0.525 120%
14 Auryn 26.482 4327% 0.543 124%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

1000 iterations, startup time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 5.15 100% 0.436 100%
2 Zen 5.28 103% 0.359 82%
3 Disco 14.011 272% 0.456 104%
4 Cobalt 20.475 398% 0.701 161%
5 PHPixie DI 20.619 400% 0.445 102%
6 Joomla 24.645 479% 0.516 118%
7 Zend ServiceManager 32.435 630% 0.515 118%
8 Dice 38.231 742% 0.61 140%
9 Pimple 52.923 1028% 0.466 107%
10 Yii2 Container 61.751 1199% 0.576 132%
11 Aura 65.715 1276% 0.626 144%
12 Opulence IoC 87.687 1703% 0.484 111%
13 Laravel 183.376 3561% 0.525 120%
14 Auryn 259.266 5034% 0.543 124%
15 PHP-DI N/A N/A N/A N/A
16 Yaco N/A N/A N/A N/A

Test Suite 5: "Warm" Fetching of the same small object graph (10 objects)

In this Test Suite, containers have to fetch an object graph of 10 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.05 100% 0.351 100%
2 Symfony 0.054 108% 0.42 120%
3 Zend ServiceManager 0.06 120% 0.462 132%
4 Yaco 0.06 120% 0.393 112%
5 PHP-DI 0.061 122% 0.354 101%
6 Aura 0.063 126% 0.46 131%
7 Yii2 Container 0.063 126% 0.487 138%
8 Dice 0.067 134% 0.382 109%
9 Cobalt 0.092 184% 0.44 125%
10 Pimple 0.123 246% 0.399 114%
11 Joomla 0.144 288% 0.43 122%
12 Opulence IoC 0.199 398% 0.442 126%
13 Disco 0.215 430% 0.445 127%
14 Laravel 0.232 464% 0.434 123%
15 PHPixie DI 0.255 510% 0.395 112%
16 Auryn 0.382 764% 0.382 109%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.494 100% 0.351 100%
2 Symfony 0.535 108% 0.42 120%
3 Zend ServiceManager 0.59 119% 0.462 132%
4 Yaco 0.598 121% 0.393 112%
5 PHP-DI 0.602 122% 0.354 101%
6 Aura 0.623 126% 0.46 131%
7 Yii2 Container 0.624 126% 0.487 138%
8 Dice 0.665 135% 0.382 109%
9 Cobalt 0.91 184% 0.44 125%
10 Pimple 1.228 249% 0.399 114%
11 Joomla 1.441 292% 0.43 122%
12 Opulence IoC 1.99 403% 0.442 126%
13 Disco 2.164 438% 0.445 127%
14 Laravel 2.329 471% 0.434 123%
15 PHPixie DI 2.554 517% 0.395 112%
16 Auryn 3.827 775% 0.382 109%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 4.95 100% 0.351 100%
2 Symfony 5.362 108% 0.42 120%
3 Zend ServiceManager 5.919 120% 0.462 132%
4 Yaco 5.992 121% 0.393 112%
5 PHP-DI 6.039 122% 0.354 101%
6 Aura 6.247 126% 0.46 131%
7 Yii2 Container 6.259 126% 0.487 138%
8 Dice 6.673 135% 0.382 109%
9 Cobalt 9.115 184% 0.44 125%
10 Pimple 12.223 247% 0.399 114%
11 Joomla 14.356 290% 0.43 122%
12 Opulence IoC 19.865 401% 0.442 126%
13 Disco 21.491 434% 0.445 127%
14 Laravel 23.23 469% 0.434 123%
15 PHPixie DI 25.459 514% 0.395 112%
16 Auryn 38.246 773% 0.382 109%

Test Suite 6: "Warm" Fetching of the same big object graph (100 objects)

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor startup time are included in the measurements.

1000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.049 100% 0.359 100%
2 Symfony 0.054 110% 0.441 123%
3 Zend ServiceManager 0.06 122% 0.485 135%
4 PHP-DI 0.061 124% 0.446 124%
5 Yaco 0.062 127% 0.411 114%
6 Aura 0.063 129% 0.651 181%
7 Yii2 Container 0.063 129% 0.584 163%
8 Dice 0.067 137% 0.705 196%
9 Cobalt 0.096 196% 0.662 184%
10 Pimple 0.123 251% 0.425 118%
11 Joomla 0.146 298% 0.448 125%
12 Opulence IoC 0.202 412% 0.549 153%
13 Disco 0.223 455% 0.488 136%
14 Laravel 0.233 476% 0.514 143%
15 PHPixie DI 0.257 524% 0.403 112%
16 Auryn 0.384 784% 0.607 169%

10 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.489 100% 0.359 100%
2 Symfony 0.535 109% 0.441 123%
3 Zend ServiceManager 0.59 121% 0.485 135%
4 PHP-DI 0.602 123% 0.446 124%
5 Yaco 0.612 125% 0.411 114%
6 Aura 0.623 127% 0.651 181%
7 Yii2 Container 0.624 128% 0.584 163%
8 Dice 0.665 136% 0.705 196%
9 Cobalt 0.956 196% 0.662 184%
10 Pimple 1.228 251% 0.425 118%
11 Joomla 1.452 297% 0.448 125%
12 Opulence IoC 2.025 414% 0.549 153%
13 Disco 2.238 458% 0.488 136%
14 Laravel 2.329 476% 0.514 143%
15 PHPixie DI 2.569 525% 0.403 112%
16 Auryn 3.838 785% 0.607 169%

100 000 iterations, bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 4.913 100% 0.359 100%
2 Symfony 5.36 109% 0.441 123%
3 Zend ServiceManager 5.915 120% 0.485 135%
4 PHP-DI 6.038 123% 0.446 124%
5 Yaco 6.142 125% 0.411 114%
6 Aura 6.245 127% 0.651 181%
7 Yii2 Container 6.266 128% 0.584 163%
8 Dice 6.67 136% 0.705 196%
9 Cobalt 9.566 195% 0.662 184%
10 Pimple 12.213 249% 0.425 118%
11 Joomla 14.532 296% 0.448 125%
12 Opulence IoC 20.19 411% 0.549 153%
13 Disco 22.204 452% 0.488 136%
14 Laravel 23.224 473% 0.514 143%
15 PHPixie DI 25.626 522% 0.403 112%
16 Auryn 38.364 781% 0.607 169%


Conclusion

Keep in mind that in a well-architected application you won't call your DI Container hundreds or even thousands of times because ideally there should be as few composition roots as possible (but there is a good chance of needing the container in other places of the application layer - e.g. in your middleware or bootstrap files). That's why most results are exaggerated - you probably won't see milliseconds of difference between the fastest and the slowest DIC in the real life.

To sum up, when choosing a container it only depends on your needs which one suits your project best: if you have a performance-critical application then you probably want to choose a compiled container. If maximum performance is not required, but you develop a big and complex system then I would recommend a dynamic container with autowiring capabilities. Otherwise you can go with simpler containers.