criterion performance measurements
overview
want to understand this report?
| int | |
| w32 | |
| inc |
int/loop
940 μs 950 960 970 980 990 1000
mean |
940 μs 950 960 970 980 990 1000
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 949.2 μs | 951.1 μs | 953.7 μs |
| Standard deviation | 9.091 μs | 11.28 μs | 14.64 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
int/unsafeLoop
630 μs 640 650 660 670 680 690
mean |
630 μs 640 650 660 670 680 690
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 632.5 μs | 633.8 μs | 636.5 μs |
| Standard deviation | 4.804 μs | 9.436 μs | 15.70 μs |
Outlying measurements have slight (7.5%) effect on estimated standard deviation.
int/numLoop
620 μs 630 640 650 660 670 680 690 700 710
mean |
620 μs 630 640 650 660 670 680 690 700 710
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 633.0 μs | 634.9 μs | 637.9 μs |
| Standard deviation | 8.348 μs | 11.89 μs | 18.06 μs |
Outlying measurements have moderate (11.4%) effect on estimated standard deviation.
int/forLoop
650 μs 700 750 800
mean |
650 μs 700 750 800
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 640.9 μs | 645.7 μs | 653.8 μs |
| Standard deviation | 21.34 μs | 31.36 μs | 47.60 μs |
Outlying measurements have moderate (46.5%) effect on estimated standard deviation.
int/loopLocal
950 μs 1000 1050 1100 1150 1200 1250
mean |
950 μs 1000 1050 1100 1150 1200 1250
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 956.7 μs | 963.2 μs | 975.0 μs |
| Standard deviation | 27.14 μs | 43.54 μs | 68.60 μs |
Outlying measurements have moderate (43.5%) effect on estimated standard deviation.
int/unsafeLoopLocal
620 μs 630 640 650 660 670 680 690 700 710
mean |
620 μs 630 640 650 660 670 680 690 700 710
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 633.3 μs | 635.2 μs | 638.2 μs |
| Standard deviation | 8.902 μs | 12.19 μs | 18.76 μs |
Outlying measurements have moderate (12.3%) effect on estimated standard deviation.
int/loopMonad
940 μs 960 980 1000 1020 1040
mean |
940 μs 960 980 1000 1020 1040
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 965.6 μs | 969.4 μs | 974.2 μs |
| Standard deviation | 18.52 μs | 21.83 μs | 26.11 μs |
Outlying measurements have moderate (16.1%) effect on estimated standard deviation.
int/numLoopMonadIntInlinable
2.75 ms 2.80 2.85 2.90
mean |
2.75 ms 2.80 2.85 2.90
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 2.792 ms | 2.798 ms | 2.805 ms |
| Standard deviation | 28.91 μs | 33.60 μs | 41.40 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
int/numLoopMonadIntInline
620 μs 640 660 680 700 720
mean |
620 μs 640 660 680 700 720
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 639.3 μs | 642.1 μs | 645.8 μs |
| Standard deviation | 13.49 μs | 16.48 μs | 23.21 μs |
Outlying measurements have moderate (20.0%) effect on estimated standard deviation.
int/numLoopInt
650 μs 700 750 800
mean |
650 μs 700 750 800
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 638.6 μs | 641.6 μs | 647.2 μs |
| Standard deviation | 13.27 μs | 20.40 μs | 37.38 μs |
Outlying measurements have moderate (26.8%) effect on estimated standard deviation.
int/numLoopEndInt
630 μs 640 650 660 670 680
mean |
630 μs 640 650 660 670 680
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 632.5 μs | 633.9 μs | 635.9 μs |
| Standard deviation | 6.574 μs | 8.588 μs | 13.47 μs |
Outlying measurements have slight (6.6%) effect on estimated standard deviation.
int/listForM_Int
620 μs 640 660 680 700 720 740
mean |
620 μs 640 660 680 700 720 740
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 634.2 μs | 636.5 μs | 641.1 μs |
| Standard deviation | 7.699 μs | 15.83 μs | 26.58 μs |
Outlying measurements have moderate (19.0%) effect on estimated standard deviation.
int/vectorFromListInt
4.2 ms 4.4 4.6 4.8 5.0 5.2
mean |
4.2 ms 4.4 4.6 4.8 5.0 5.2
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 4.311 ms | 4.341 ms | 4.383 ms |
| Standard deviation | 139.1 μs | 179.7 μs | 241.0 μs |
Outlying measurements have moderate (38.5%) effect on estimated standard deviation.
int/vectorEnumFromToInt
600 μs 700 800 900 1000 1100
mean |
600 μs 700 800 900 1000 1100
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 640.1 μs | 646.0 μs | 668.0 μs |
| Standard deviation | 13.10 μs | 52.07 μs | 120.5 μs |
Outlying measurements have severe (70.8%) effect on estimated standard deviation.
int/uvectorFromListInt
4.20 ms 4.30 4.40 4.50 4.60 4.70
mean |
4.20 ms 4.30 4.40 4.50 4.60 4.70
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 4.265 ms | 4.287 ms | 4.314 ms |
| Standard deviation | 99.56 μs | 122.0 μs | 147.5 μs |
Outlying measurements have moderate (22.9%) effect on estimated standard deviation.
int/uvectorEnumFromToInt
630 μs 640 650 660 670 680
mean |
630 μs 640 650 660 670 680
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 636.8 μs | 638.9 μs | 641.7 μs |
| Standard deviation | 10.06 μs | 12.40 μs | 15.38 μs |
Outlying measurements have moderate (12.3%) effect on estimated standard deviation.
int/FoldL.FoldM
630 μs 640 650 660 670 680 690
mean |
630 μs 640 650 660 670 680 690
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 647.5 μs | 650.6 μs | 654.0 μs |
| Standard deviation | 15.03 μs | 16.74 μs | 18.57 μs |
Outlying measurements have moderate (20.0%) effect on estimated standard deviation.
w32/loop
940 μs 960 980 1000 1020
mean |
940 μs 960 980 1000 1020
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 958.6 μs | 963.1 μs | 968.5 μs |
| Standard deviation | 21.86 μs | 25.37 μs | 28.93 μs |
Outlying measurements have moderate (20.0%) effect on estimated standard deviation.
w32/unsafeLoop
940 μs 960 980 1000 1020 1040
mean |
940 μs 960 980 1000 1020 1040
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 978.6 μs | 985.6 μs | 992.9 μs |
| Standard deviation | 34.81 μs | 36.77 μs | 38.94 μs |
Outlying measurements have moderate (33.6%) effect on estimated standard deviation.
w32/numLoop
630 μs 640 650 660 670
mean |
630 μs 640 650 660 670
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 633.6 μs | 635.5 μs | 638.1 μs |
| Standard deviation | 9.227 μs | 11.39 μs | 14.02 μs |
Outlying measurements have moderate (10.4%) effect on estimated standard deviation.
w32/forLoop
620 μs 640 660 680 700 720 740
mean |
620 μs 640 660 680 700 720 740
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 636.3 μs | 639.0 μs | 643.3 μs |
| Standard deviation | 12.49 μs | 17.10 μs | 26.93 μs |
Outlying measurements have moderate (20.9%) effect on estimated standard deviation.
w32/loopLocal
940 μs 950 960 970 980 990 1000 1010
mean |
940 μs 950 960 970 980 990 1000 1010
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 951.2 μs | 953.6 μs | 956.7 μs |
| Standard deviation | 11.54 μs | 14.00 μs | 18.20 μs |
Outlying measurements have slight (7.5%) effect on estimated standard deviation.
w32/unsafeLoopLocal
940 μs 960 980 1000 1020 1040
mean |
940 μs 960 980 1000 1020 1040
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 963.8 μs | 968.0 μs | 973.0 μs |
| Standard deviation | 20.34 μs | 23.53 μs | 27.36 μs |
Outlying measurements have moderate (18.0%) effect on estimated standard deviation.
w32/loopMonad
0.95 ms 1.00 1.05 1.10 1.15 1.20 1.25 1.30
mean |
0.95 ms 1.00 1.05 1.10 1.15 1.20 1.25 1.30
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 1.004 ms | 1.018 ms | 1.035 ms |
| Standard deviation | 68.97 μs | 80.77 μs | 96.78 μs |
Outlying measurements have severe (70.7%) effect on estimated standard deviation.
w32/numLoopW32
630 μs 640 650 660 670 680
mean |
630 μs 640 650 660 670 680
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 637.9 μs | 640.2 μs | 642.9 μs |
| Standard deviation | 10.88 μs | 12.88 μs | 15.45 μs |
Outlying measurements have moderate (13.2%) effect on estimated standard deviation.
w32/numLoopEndW32
625 μs 630 635 640 645 650 655 660 665 670
mean |
625 μs 630 635 640 645 650 655 660 665 670
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 633.4 μs | 634.9 μs | 636.7 μs |
| Standard deviation | 6.968 μs | 8.402 μs | 10.43 μs |
Outlying measurements have slight (6.6%) effect on estimated standard deviation.
w32/listForM_W32
4.80 ms 4.90 5.00 5.10 5.20 5.30
mean |
4.80 ms 4.90 5.00 5.10 5.20 5.30
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.048 ms | 5.063 ms | 5.079 ms |
| Standard deviation | 66.10 μs | 78.66 μs | 96.43 μs |
Outlying measurements have slight (8.5%) effect on estimated standard deviation.
w32/vectorFromListW32
4.9 ms 5.0 5.1 5.2 5.3 5.4 5.5 5.6
mean |
4.9 ms 5.0 5.1 5.2 5.3 5.4 5.5 5.6
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.073 ms | 5.093 ms | 5.118 ms |
| Standard deviation | 91.37 μs | 114.0 μs | 144.1 μs |
Outlying measurements have moderate (16.1%) effect on estimated standard deviation.
w32/vectorEnumFromToW32
620 μs 640 660 680 700 720
mean |
620 μs 640 660 680 700 720
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 642.3 μs | 645.3 μs | 649.2 μs |
| Standard deviation | 14.20 μs | 17.45 μs | 22.83 μs |
Outlying measurements have moderate (21.0%) effect on estimated standard deviation.
w32/uvectorFromListW32
4.8 ms 4.9 5.0 5.1 5.2 5.3 5.4
mean |
4.8 ms 4.9 5.0 5.1 5.2 5.3 5.4
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.055 ms | 5.070 ms | 5.086 ms |
| Standard deviation | 64.90 μs | 80.16 μs | 102.8 μs |
Outlying measurements have slight (8.5%) effect on estimated standard deviation.
w32/uvectorEnumFromToW32
620 μs 640 660 680 700 720
mean |
620 μs 640 660 680 700 720
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 633.7 μs | 635.7 μs | 639.2 μs |
| Standard deviation | 8.991 μs | 13.36 μs | 21.80 μs |
Outlying measurements have moderate (14.2%) effect on estimated standard deviation.
w32/FoldL.FoldM
4.8 ms 4.9 5.0 5.1 5.2 5.3 5.4 5.5
mean |
4.8 ms 4.9 5.0 5.1 5.2 5.3 5.4 5.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.032 ms | 5.053 ms | 5.081 ms |
| Standard deviation | 98.81 μs | 124.4 μs | 156.7 μs |
Outlying measurements have moderate (18.1%) effect on estimated standard deviation.
inc/+1
10.2 ns 10.4 10.6 10.8 11.0
mean |
10.2 ns 10.4 10.6 10.8 11.0
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 10.24 ns | 10.27 ns | 10.31 ns |
| Standard deviation | 127.9 ps | 159.9 ps | 222.7 ps |
Outlying measurements have slight (8.5%) effect on estimated standard deviation.
inc/succ
10.2 ns 10.3 10.4 10.5 10.6 10.7 10.8 10.9
mean |
10.2 ns 10.3 10.4 10.5 10.6 10.7 10.8 10.9
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 10.34 ns | 10.37 ns | 10.40 ns |
| Standard deviation | 122.5 ps | 145.9 ps | 176.7 ps |
Outlying measurements have slight (6.6%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. In each section, we display two charts, each with an x axis that represents measured execution time. These charts are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. Measurements are displayed on the y axis in the order in which they occurred.
Under the charts is a small table displaying the mean and standard deviation of the measurements. We use a statistical technique called the bootstrap to provide confidence intervals on our estimates of these values. The bootstrap-derived upper and lower bounds on the mean and standard deviation let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.