namespace inner

analytics~metrics

Classification and regression metrics.

Examples

Batch classification example

// import metrics module
var analytics = require('qminer').analytics;

// true and predicted lables
var true_lables = [0, 1, 0, 0, 1];
var pred_prob = [0.3, 0.5, 0.2, 0.5, 0.8];

// compute ROC curve
var roc = analytics.metrics.rocCurve(true_lables, pred_prob);

Online classification example

// import analytics module
var analytics = require('qminer').analytics;
// true and predicted lables
var true_lables = [0, 1, 0, 0, 1];
var pred_prob = [0.3, 0.5, 0.2, 0.5, 0.8];

// create predictionCurve instance
var predictionCurve = new analytics.metrics.PredictionCurve();

// simulate data flow
for (var i in true_lables) {
   // push new value
   predictionCurve.push(true_lables[i], pred_prob[i]);
}

var roc = predictionCurve.roc(); // get ROC

Batch regression example

// import analytics module
var analytics = require('qminer').analytics;
// true and predicted data
var true_vals = [1, 2, 3, 4, 5];
var pred_vals = [3, 4, 5, 6, 7];

// use batch MAE method
analytics.metrics.meanAbsoluteError(true_vals, pred_vals);

Online regression example

// import analytics module
var analytics = require('qminer').analytics;
// true and predicted data
var true_vals = [1, 2, 3, 4, 5];
var pred_vals = [3, 4, 5, 6, 7];

// create online MAE metric instance
var mae = new analytics.metrics.MeanAbsoluteError();

// simulate data flow
for (var i in true_vals) {
  // push new value
  mae.push(true_vals[i], pred_vals[i]);
}
// get updated error
mae.getError();

Child classes

ClassificationScore(yTrue, yPred)
MeanAbsoluteError([fin])
MeanAbsolutePercentageError([fin])

MeanError([fin])
MeanSquareError([fin])
PredictionCurve(yTrue, yPred)

R2Score([fin])
RootMeanSquareError([fin])

Methods

accuracyScore(yTrue, yPred)
bestF1Threshold(yTrue, yPred)
breakEventPointScore(yTrue, yPred)
desiredPrecisionThreshold(yTrue, yPred, desiredPrecision)
desiredRecallThreshold(yTrue, yPred, desiredRecall)
f1Score(yTrue, yPred)

meanAbsoluteError(yTrueVec, yPredVec)
meanAbsolutePercentageError(yTrueVec, yPredVec)
meanError(yTrueVec, yPredVec)
meanSquareError(yTrueVec, yPredVec)
precisionRecallCurve(yTrue, yPred[, sample])
precisionScore(yTrue, yPred)

r2Score(yTrueVec, yPredVec)
recallScore(yTrue, yPred)
rocAucScore(yTrue, yPred[, sample])
rocCurve(yTrue, yPred[, sample])
rootMeanSquareError(yTrueVec, yPredVec)

Classes

Methods

static

accuracyScore(yTrue, yPred) → number

Accuracy score is the proportion of true results (both true positives and true negatives) among the total number of cases examined. Formula: (tp + tn) / (tp + fp + fn + tn).

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Predicted (estimated) lables.

Returns: numberB Accuracy value.

static

bestF1Threshold(yTrue, yPred) → number

Gets threshold for prediction score, which results in the highest F1.

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Estimated probabilities.

Returns: numberB Threshold with highest F1 score.

static

breakEventPointScore(yTrue, yPred) → number

Get break-even point, the value where precision and recall intersect.

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Estimated probabilities.

Returns: numberB Break-even point score.

static

desiredPrecisionThreshold(yTrue, yPred, desiredPrecision) → number

Gets threshold for prediction score, nearest to specified precision.

Parameters

Name	Type	Description
yTrue	(Array of number or module:la.Vector)	Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)	Estimated probabilities.
desiredPrecision	number	Desired precision score.

Returns: numberB Threshold for prediction score, nearest to specified precision.

static

desiredRecallThreshold(yTrue, yPred, desiredRecall) → number

Gets threshold for recall score, nearest to specified recall.

Parameters

Name	Type	Description
yTrue	(Array of number or module:la.Vector)	Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)	Estimated probabilities.
desiredRecall	number	Desired recall score.

Returns: numberB Threshold for recall score, nearest to specified recall.

static

f1Score(yTrue, yPred) → number

The F1 score can be interpreted as a weighted average of the precision and recall, where an F1 score reaches its best value at 1 and worst score at 0. The relative contribution of precision and recall to the F1 score are equal. Formula: 2 * (precision * recall) / (precision + recall).

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Predicted (estimated) lables.

Returns: numberB F1 score.

static

meanAbsoluteError(yTrueVec, yPredVec) → number

Mean absolute error (MAE) regression loss.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec`.

Returns: numberB Error value.

static

meanAbsolutePercentageError(yTrueVec, yPredVec) → number

Mean absolute percentage error (MAPE) regression loss.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec.`

Returns: numberB Error value.

static

meanError(yTrueVec, yPredVec) → number

Mean error (ME) regression loss.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec`.

Returns: numberB Error value.

static

meanSquareError(yTrueVec, yPredVec) → number

Mean square error (MSE) regression loss.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec`.

Returns: numberB Error value.

static

precisionRecallCurve(yTrue, yPred[, sample]) → module:la.Matrix

Get precision recall curve sampled on sample points.

Parameters

Name Type Optional Description

yTrue

(Array of number or module:la.Vector)

Ground truth (correct) lables.

yPred

(Array of number or module:la.Vector)

Estimated probabilities.

sample

number

Yes

Desired number of samples in output.

Defaults to 10.

Returns: module:la.MatrixB Precision-recall pairs.

static

precisionScore(yTrue, yPred) → number

Precision score is defined as the proportion of the true positives against all the positive results (both true positives and false positives). Formula: tp / (tp + fp).

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Predicted (estimated) lables.

Returns: numberB Precission score.

static

r2Score(yTrueVec, yPredVec) → number

R^2 (coefficient of determination) regression score.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec`.

Returns: numberB Error value.

static

recallScore(yTrue, yPred) → number

Recall score is intuitively the ability of the classifier to find all the positive samples. Formula: tp / (tp + fn).

Parameters

Name	Type	Optional	Description
yTrue	(Array of number or module:la.Vector)		Ground truth (correct) lables.
yPred	(Array of number or module:la.Vector)		Predicted (estimated) lables.

Returns: numberB Recall score.

static

rocAucScore(yTrue, yPred[, sample]) → number

Get AUC of the current curve.

Example

// import metrics module
var metrics = require('qminer').analytics.metrics;

// true and predicted lables
var true_lables = [0, 1, 0, 0, 1];
var pred_prob = [0.3, 0.5, 0.2, 0.5, 0.8];

// compute ROC curve
var auc = metrics.rocAucScore(true_lables, pred_prob); // output: 0.92

Parameters

Name Type Optional Description

yTrue

(Array of number or module:la.Vector)

Ground truth (correct) lables.

yPred

(Array of number or module:la.Vector)

Estimated probabilities.

sample

number

Yes

Desired number of samples in output.

Defaults to 10.

Returns: numberB Area under ROC curve.

static

rocCurve(yTrue, yPred[, sample]) → module:la.Matrix

Get ROC parametrization sampled on sample points.

Example

// import metrics module
var metrics = require('qminer').analytics.metrics;

// true and predicted lables
var true_lables = [0, 1, 0, 0, 1];
var pred_prob = [0.3, 0.5, 0.2, 0.5, 0.8];

// compute ROC curve
var roc = metrics.rocCurve(true_lables, pred_prob); // output: [ [ 0, 0 ], [0, 0.5], [[ 0.34, 1 ],], [ 0.67, 0 ], [ 1, 1 ] ]

Parameters

Name Type Optional Description

yTrue

(Array of number or module:la.Vector)

Ground truth (correct) lables.

yPred

(Array of number or module:la.Vector)

Estimated probabilities.

sample

number

Yes

Desired number of samples in output.

Defaults to 10.

Returns: module:la.MatrixB A matrix with increasing false and true positive rates.

static

rootMeanSquareError(yTrueVec, yPredVec) → number

Root mean square (RMSE) error regression loss.

Parameters

Name	Type	Optional	Description
yTrueVec	(Array of number or module:la.Vector)		ground truth values in `yTrueVec`.
yPredVec	(Array of number or module:la.Vector)		estimated values in `yPredVec`.

Returns: numberB Error value.