class static

analytics.TDigest

TDigest is a methods that approximates the CDF function of streaming measurements. Data structure useful for percentile and quantile estimation for online data streams. It can be added to any anomaly detector to set the number of alarms triggered as a percentage of the total samples. Adding new samples to the distribution is achieved through insert and querying the model (computing quantiles) is implemented by predict function.
This is based on the Data Lib Sketch Implementation: t-digest.js
Paper: Ted Dunning, Otmar Ertl - Computing Extremely Accurate Quantiles Using t-Digests.

Properties

memory

size

init

Methods

getParams()
insert(x)

quantile(x)
save(fout)

new TDigest([arg])

TDigest quantile estimation on streams

Example

// import modules
var qm = require('qminer');
var fs = qm.fs;
var analytics = qm.analytics;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// create the data used for calculating quantiles
var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3];
// fit the TDigest model
for (var i = 0; i < inputs.length; i++) {
    tdigest.insert(inputs[i]);
}
// make the prediction for the 0.1 quantile
var prediction = tdigest.quantile(0.1);
// save the model
tdigest.save(fs.openWrite('tdigest.bin')).close();
// open the tdigest model under a new variable
var tdigest2 = new analytics.quantiles.TDigest(fs.openRead('tdigest.bin'));

Parameter

Name	Type	Optional	Description
arg	(module:analytics.quantiles~TDigestParam or module:fs.FIn)	Yes	Construction arguments. There are two ways of constructing: 1. Using the module:analytics.quantiles~TDigestParam object, 2. using the file input stream module:fs.FIn.

Properties

static

memory

Returns the models current memory consumption.

static

size

Returns the current size of the algorithms summary in number of tuples.

init

Returns true when the model has enough data to initialize. Type boolean.

Example

// import modules
var qm = require('qminer');
var analytics = qm.analytics;
var fs = qm.fs;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// check if the model has enough data to initialize
if (tdigest.init) { console.log("Ready to initialize"); }

Methods

getParams() → module:analytics.quantiles~TDigestParam

Returns the parameters.

Example

// import modules
var qm = require('qminer');
var analytics = qm.analytics;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// get the parameters of the object
var params = tdigest.getParams();

Returns: module:analytics.quantiles~TDigestParamB The construction parameters.

insert(x) → module:analytics.quantiles.TDigest

Adds a new measurement to the model and updates the approximation of the data distribution.

Example

// import modules
var qm = require('qminer');
var analytics = qm.analytics;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// create the data used for calculating quantiles
var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3];
// fit the TDigest model with all input values
for (var i = 0; i < inputs.length; i++) {
    tdigest.insert(inputs[i]);
}

Parameter

Name	Type	Optional	Description
x	number		Input number.

Returns: module:analytics.quantiles.TDigestB Self. The model has been updated.

quantile(x) → number

Returns a quantile given input number, that is the approximate fraction of samples smaller than the input (0.05 means that 5% of data is smaller than the input value).

Example

// import modules
var qm = require('qminer');
var analytics = qm.analytics;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// create the data used for calculating quantiles
var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3];
// fit the TDigest model
for (var i = 0; i < inputs.length; i++) {
    tdigest.insert(inputs[i]);
}
// make the estimation for the 0.1 quantile
var quant = tdigest.quantile(0.1);

Parameter

Name	Type	Optional	Description
x	number		Input number.

Returns: numberB Quantile (between 0 and 1).

save(fout) → module:fs.FOut

Saves TDigest internal state into (binary) file.

Example

// import modules
var qm = require('qminer');
var fs = require('qminer').fs;
var analytics = qm.analytics;
var fs = qm.fs;
// create the default TDigest object
var tdigest = new analytics.quantiles.TDigest();
// create the data used for calculating quantiles
var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3];
// fit the TDigest model
for (var i = 0; i < inputs.length; i++) {
    tdigest.insert(inputs[i]);
}
// save the model
tdigest.save(fs.openWrite('tdigest.bin')).close();
// open the tdigest model under a new variable
var tdigest2 = new analytics.quantiles.TDigest(fs.openRead('tdigest.bin'));

Parameter

Name	Type	Optional	Description
fout	module:fs.FOut		The output stream.

Returns: module:fs.FOutB The output stream fout.