FastLinearClassifier Class

Inheritance

nimbusml.internal.core.linear_model._fastlinearclassifier.FastLinearClassifier

FastLinearClassifier

nimbusml.base_predictor.BasePredictor

FastLinearClassifier

sklearn.base.ClassifierMixin

FastLinearClassifier

Constructor

FastLinearClassifier(l2_regularization=None, l1_threshold=None, normalize='Auto', caching='Auto', loss='log', number_of_threads=None, convergence_tolerance=0.1, maximum_number_of_iterations=None, shuffle=True, convergence_check_frequency=None, bias_learning_rate=0.0, feature=None, label=None, weight=None, **params)

Parameters

Examples

   ###############################################################################
   # FastLinearClassifier
   from nimbusml import Pipeline, FileDataStream
   from nimbusml.datasets import get_dataset
   from nimbusml.feature_extraction.categorical import OneHotVectorizer
   from nimbusml.linear_model import FastLinearClassifier

   # data input (as a FileDataStream)
   path = get_dataset('infert').as_filepath()
   data = FileDataStream.read_csv(path)
   print(data.head())
   #   age  case education  induced  parity  ... row_num  spontaneous  ...
   # 0   26     1    0-5yrs        1       6 ...       1            2  ...
   # 1   42     1    0-5yrs        1       1 ...       2            0  ...
   # 2   39     1    0-5yrs        2       6 ...       3            0  ...
   # 3   34     1    0-5yrs        2       4 ...       4            0  ...
   # 4   35     1   6-11yrs        1       3 ...       5            1  ...


   # define the training pipeline
   pipeline = Pipeline([
       OneHotVectorizer(columns={'edu': 'education'}),
       FastLinearClassifier(feature=['age', 'edu', 'parity'], label='induced')
   ])

   # train, predict, and evaluate
   metrics, predictions = pipeline.fit(data).test(data, output_scores=True)

   # print predictions
   print(predictions.head())
   #   PredictedLabel   Score.0   Score.1   Score.2
   # 0               2  0.015312  0.058199  0.926489
   # 1               0  0.892915  0.097093  0.009991
   # 2               2  0.058976  0.123581  0.817444
   # 3               2  0.287882  0.245397  0.466721
   # 4               0  0.404075  0.362293  0.233632

   # print evaluation metrics
   print(metrics)
   #   Accuracy(micro-avg)  Accuracy(macro-avg)  Log-loss  Log-loss reduction  ...
   # 0             0.604839             0.489888  0.857278
   # 10.336077  ...

Remarks

FastLinearClassifier is a trainer based on the Stochastic Dual Coordinate Ascent (SDCA) method, a state-of-the-art optimization technique for convex objective functions. The algorithm can be scaled for use on large out-of-memory data sets due to a semi-asynchronized implementation that supports multi-threading. Convergence is underwritten by periodically enforcing synchronization between primal and dual updates in a separate thread. Several choices of loss functions are also provided. The SDCA method combines several of the best properties and capabilities of logistic regression and SVM algorithms. For more information on SDCA, see the citations in the reference section.

Traditional optimization algorithms, such as stochastic gradient descent (SGD), optimize the empirical loss function directly. The SDCA chooses a different approach that optimizes the dual problem instead. The dual loss function is parameterized by per-example weights. In each iteration, when a training example from the training data set is read, the corresponding example weight is adjusted so that the dual loss function is optimized with respect to the current example. No learning rate is needed by SDCA to determine step size as is required by various gradient descent methods.

FastLinearClassifier supports classification with three types of loss functions currently: Log loss, hinge loss, and smoothed hinge loss. Elastic net regularization can be specified by the l2_weight and l1_threshold parameters. Note that the l2_weight has an effect on the rate of convergence. In general, the larger the l2_weight, the faster SDCA converges.

Note that FastLinearClassifier is a stochastic and streaming optimization algorithm. The results depends on the order of the training data. For reproducible results, it is recommended that one sets shuffle to False and number_of_threads to 1.

Reference

Scaling Up Stochastic Dual Coordinate Ascent

Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization

Methods

decision_function

decision_function(X, **params)

get_params

get_params(deep=False)

predict_proba

predict_proba(X, **params)