criterion performance measurements
overview
want to understand this report?
sum32/sumW32loopIORef
5.0 ms 5.5 6.0 6.5 7.0 7.5 8.0
mean |
5.0 ms 5.5 6.0 6.5 7.0 7.5 8.0
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.437 ms | 5.501 ms | 5.612 ms |
| Standard deviation | 280.2 μs | 422.8 μs | 649.0 μs |
Outlying measurements have severe (68.7%) effect on estimated standard deviation.
sum32/sumW32StrictState
0.95 ms 1.00 1.05 1.10 1.15 1.20
mean |
0.95 ms 1.00 1.05 1.10 1.15 1.20
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 1.016 ms | 1.030 ms | 1.044 ms |
| Standard deviation | 66.61 μs | 73.90 μs | 84.26 μs |
Outlying measurements have severe (65.6%) effect on estimated standard deviation.
sum32/foldlW32
13.0 ms 13.2 13.4 13.6 13.8 14.0
mean |
13.0 ms 13.2 13.4 13.6 13.8 14.0
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 13.09 ms | 13.11 ms | 13.15 ms |
| Standard deviation | 91.05 μs | 153.7 μs | 241.4 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
sum32/V.foldlW32
13.2 ms 13.4 13.6 13.8 14.0 14.2
mean |
13.2 ms 13.4 13.6 13.8 14.0 14.2
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 13.27 ms | 13.29 ms | 13.33 ms |
| Standard deviation | 82.76 μs | 138.0 μs | 241.6 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
sum32/sumIntloopIORef
5.175 ms 5.200 5.225 5.250 5.275 5.300 5.325
mean |
5.175 ms 5.200 5.225 5.250 5.275 5.300 5.325
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 5.198 ms | 5.203 ms | 5.210 ms |
| Standard deviation | 23.01 μs | 30.48 μs | 39.68 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
sum32/sumIntStrictState
940 μs 950 960 970 980 990 1000 1010
mean |
940 μs 950 960 970 980 990 1000 1010
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 944.1 μs | 945.3 μs | 947.7 μs |
| Standard deviation | 5.343 μs | 8.343 μs | 15.24 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
sum32/foldlInt
9.1 ms 9.2 9.3 9.4 9.5 9.6 9.7 9.8
mean |
9.1 ms 9.2 9.3 9.4 9.5 9.6 9.7 9.8
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 9.204 ms | 9.223 ms | 9.256 ms |
| Standard deviation | 75.00 μs | 123.5 μs | 180.8 μs |
Outlying measurements have slight (6.6%) effect on estimated standard deviation.
sum32/V.foldlInt
627.5 μs 630.0 632.5 635.0 637.5 640.0 642.5
mean |
627.5 μs 630.0 632.5 635.0 637.5 640.0 642.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 628.7 μs | 629.3 μs | 630.1 μs |
| Standard deviation | 2.778 μs | 3.406 μs | 4.153 μs |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. In each section, we display two charts, each with an x axis that represents measured execution time. These charts are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. Measurements are displayed on the y axis in the order in which they occurred.
Under the charts is a small table displaying the mean and standard deviation of the measurements. We use a statistical technique called the bootstrap to provide confidence intervals on our estimates of these values. The bootstrap-derived upper and lower bounds on the mean and standard deviation let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.