abacusai.model_monitor — AbacusAI documentation

# abacusai.model_monitor

## Classes

`ModelMonitorVersion` | A version of a model monitor
`RefreshSchedule` | A refresh schedule for an object. Defines when the next version of the object will be created
`AbstractApiClass` |
`ModelMonitor` | A model monitor

## Module Contents

class abacusai.model_monitor.ModelMonitorVersion(client, modelMonitorVersion=None, status=None, modelMonitorId=None, monitoringStartedAt=None, monitoringCompletedAt=None, trainingFeatureGroupVersion=None, predictionFeatureGroupVersion=None, error=None, pendingDeploymentIds=None, failedDeploymentIds=None, metricConfigs=None, featureGroupMonitorConfigs=None, metricTypes=None, modelVersion=None, batchPredictionVersion=None, edaConfigs=None, trainingForecastConfig=None, predictionForecastConfig=None, forecastFrequency=None, monitorDriftConfig=None, predictionDataUseMappings=None, trainingDataUseMappings=None)

  Bases: `abacusai.return_class.AbstractApiClass`

  A version of a model monitor

  Parameters:

    - client (ApiClient) – An authenticated API Client instance

    - modelMonitorVersion (str) – The unique identifier of a model monitor version.

    - status (str) – The current status of the model.

    - modelMonitorId (str) – A reference to the model monitor this version belongs to.

    - monitoringStartedAt (str) – The start time and date of the monitoring process.

    - monitoringCompletedAt (str) – The end time and date of the monitoring process.

    - trainingFeatureGroupVersion (list[str]) – Feature group version IDs that this refresh pipeline run is monitoring.

    - predictionFeatureGroupVersion (list[str]) – Feature group version IDs that this refresh pipeline run is monitoring.

    - error (str) – Relevant error if the status is FAILED.

    - pendingDeploymentIds (list) – List of deployment IDs where deployment is pending.

    - failedDeploymentIds (list) – List of failed deployment IDs.

    - metricConfigs (list[dict]) – List of metric configs for the model monitor instance.

    - featureGroupMonitorConfigs (dict) – Configurations for feature group monitor

    - metricTypes (list) – List of metric types.

    - modelVersion (list[str]) – Model version IDs that this refresh pipeline run is monitoring.

    - batchPredictionVersion (str) – The batch prediction version this model monitor is monitoring

    - edaConfigs (list) – The list of eda configs for the version

    - trainingForecastConfig (dict) – The training forecast config for the monitor version

    - predictionForecastConfig (dict) – The prediction forecast config for the monitor version

    - forecastFrequency (str) – The forecast frequency for the monitor version

    - monitorDriftConfig (dict) – The monitor drift config for the monitor version

    - predictionDataUseMappings (dict) – The mapping of prediction data use to feature group version

    - trainingDataUseMappings (dict) – The mapping of training data use to feature group version

  __repr__()

    Return repr(self).

  to_dict()

    Get a dict representation of the parameters in this class

    Returns:

      The dict value representation of the class parameters

    Return type:

      dict

  get_prediction_drift()

    Gets the label and prediction drifts for a model monitor.

    Parameters:

      model_monitor_version (str) – Unique string identifier for a model monitor version created under the project.

    Returns:

      Object describing training and prediction output label and prediction distributions.

    Return type:

      DriftDistributions

  refresh()

    Calls describe and refreshes the current object’s fields

    Returns:

      The current object

    Return type:

      ModelMonitorVersion

  describe()

    Retrieves a full description of the specified model monitor version.

    Parameters:

      model_monitor_version (str) – The unique version ID of the model monitor version.

    Returns:

      A model monitor version.

    Return type:

      ModelMonitorVersion

  delete()

    Deletes the specified model monitor version.

    Parameters:

      model_monitor_version (str) – Unique identifier of the model monitor version to delete.

  metric_data(metric_type, actual_values_to_detail=None)

    Provides the data needed for decile metrics associated with the model monitor.

    Parameters:

      - metric_type (str) – The type of metric to get data for.

      - actual_values_to_detail (list) – The actual values to detail.

    Returns:

      Data associated with the metric.

    Return type:

      ModelMonitorVersionMetricData

  list_monitor_alert_versions_for_monitor_version()

    Retrieves the list of monitor alert versions for a specified monitor instance.

    Parameters:

      model_monitor_version (str) – The unique ID associated with the model monitor.

    Returns:

      A list of monitor alert versions.

    Return type:

      list[MonitorAlertVersion]

  get_drift_for_feature(feature_name, nested_feature_name=None)

    Gets the feature drift associated with a single feature in an output feature group from a prediction.

    Parameters:

      - feature_name (str) – Name of the feature to view the distribution of.

      - nested_feature_name (str) – Optionally, the name of the nested feature that the feature is in.

    Returns:

      An object describing the training and prediction output feature distributions.

    Return type:

      FeatureDistribution

  get_outliers_for_feature(feature_name=None, nested_feature_name=None)

    Gets a list of outliers measured by a single feature (or overall) in an output feature group from a prediction.

    Parameters:

      - feature_name (str) – Name of the feature to view the distribution of.

      - nested_feature_name (str) – Optionally, the name of the nested feature that the feature is in.

  wait_for_monitor(timeout=1200)

    A waiting call until model monitor version is ready.

    Parameters:

      timeout (int) – The waiting time given to the call to finish, if it doesn’t finish by the allocated time, the call is said to be timed out.

  get_status()

    Gets the status of the model monitor version.

    Returns:

      A string describing the status of the model monitor version, for e.g., pending, complete, etc.

    Return type:

      str

class abacusai.model_monitor.RefreshSchedule(client, refreshPolicyId=None, nextRunTime=None, cron=None, refreshType=None, error=None)

  Bases: `abacusai.return_class.AbstractApiClass`

  A refresh schedule for an object. Defines when the next version of the object will be created

  Parameters:

    - client (ApiClient) – An authenticated API Client instance

    - refreshPolicyId (str) – The unique identifier of the refresh policy

    - nextRunTime (str) – The next run time of the refresh policy. If null, the policy is paused.

    - cron (str) – A cron-style string that describes the when this refresh policy is to be executed in UTC

    - refreshType (str) – The type of refresh that will be run

    - error (str) – An error message for the last pipeline run of a policy

  __repr__()

    Return repr(self).

  to_dict()

    Get a dict representation of the parameters in this class

    Returns:

      The dict value representation of the class parameters

    Return type:

      dict

class abacusai.model_monitor.AbstractApiClass(client, id)
  __eq__(other)

    Return self==value.

  _get_attribute_as_dict(attribute)

class abacusai.model_monitor.ModelMonitor(client, modelMonitorId=None, name=None, createdAt=None, projectId=None, trainingFeatureGroupId=None, predictionFeatureGroupId=None, predictionFeatureGroupVersion=None, trainingFeatureGroupVersion=None, alertConfig=None, biasMetricId=None, metricConfigs=None, featureGroupMonitorConfigs=None, metricTypes=None, modelId=None, starred=None, batchPredictionId=None, monitorType=None, edaConfigs=None, trainingForecastConfig=None, predictionForecastConfig=None, forecastFrequency=None, trainingFeatureGroupSampling=None, predictionFeatureGroupSampling=None, monitorDriftConfig=None, predictionDataUseMappings=None, trainingDataUseMappings=None, refreshSchedules={}, latestMonitorModelVersion={})

  Bases: `abacusai.return_class.AbstractApiClass`

  A model monitor

  Parameters:

    - client (ApiClient) – An authenticated API Client instance

    - modelMonitorId (str) – The unique identifier of the model monitor.

    - name (str) – The user-friendly name for the model monitor.

    - createdAt (str) – Date and time at which the model was created.

    - projectId (str) – The project this model belongs to.

    - trainingFeatureGroupId (list[str]) – Feature group IDs that this model monitor is monitoring.

    - predictionFeatureGroupId (list[str]) – Feature group IDs that this model monitor is monitoring.

    - predictionFeatureGroupVersion (list[str]) – Feature group versions that this model monitor is monitoring.

    - trainingFeatureGroupVersion (list[str]) – Feature group versions that this model monitor is monitoring.

    - alertConfig (dict) – Alerting configuration for this model monitor.

    - biasMetricId (str) – The bias metric ID

    - metricConfigs (dict) – Configurations for model monitor

    - featureGroupMonitorConfigs (dict) – Configurations for feature group monitor

    - metricTypes (dict) – List of metric types

    - modelId (str) – Model ID that this model monitor is monitoring.

    - starred (bool) – Whether this model monitor is starred.

    - batchPredictionId (str) – The batch prediction ID this model monitor monitors

    - monitorType (str) – The type of the monitor, one of MODEL_MONITOR, or FEATURE_GROUP_MONITOR

    - edaConfigs (dict) – The configs for EDA

    - trainingForecastConfig (dict) – The tarining config for forecast monitors

    - predictionForecastConfig (dict) – The prediction config for forecast monitors

    - forecastFrequency (str) – The frequency of the forecast

    - trainingFeatureGroupSampling (bool) – Whether or not we sample from training feature group

    - predictionFeatureGroupSampling (bool) – Whether or not we sample from prediction feature group

    - monitorDriftConfig (dict) – The monitor drift config for the monitor

    - predictionDataUseMappings (dict) – The data_use mapping of the prediction features

    - trainingDataUseMappings (dict) – The data_use mapping of the training features

    - latestMonitorModelVersion (ModelMonitorVersion) – The latest model monitor version.

    - refreshSchedules (RefreshSchedule) – List of refresh schedules that indicate when the next model version will be trained.

  __repr__()

    Return repr(self).

  to_dict()

    Get a dict representation of the parameters in this class

    Returns:

      The dict value representation of the class parameters

    Return type:

      dict

  rerun()

    Re-runs the specified model monitor.

    Parameters:

      model_monitor_id (str) – Unique string identifier of the model monitor to re-run.

    Returns:

      The model monitor that is being re-run.

    Return type:

      ModelMonitor

  refresh()

    Calls describe and refreshes the current object’s fields

    Returns:

      The current object

    Return type:

      ModelMonitor

  describe()

    Retrieves a full description of the specified model monitor.

    Parameters:

      model_monitor_id (str) – Unique string identifier associated with the model monitor.

    Returns:

      Description of the model monitor.

    Return type:

      ModelMonitor

  get_summary()

    Gets the summary of a model monitor across versions.

    Parameters:

      model_monitor_id (str) – A unique string identifier associated with the model monitor.

    Returns:

      An object describing integrity, bias violations, model accuracy and drift for the model monitor.

    Return type:

      ModelMonitorSummary

  list_versions(limit=100, start_after_version=None)

    Retrieves a list of versions for a given model monitor.

    Parameters:

      - limit (int) – The maximum length of the list of all model monitor versions.

      - start_after_version (str) – The ID of the version after which the list starts.

    Returns:

      A list of model monitor versions.

    Return type:

      list[ModelMonitorVersion]

  rename(name)

    Renames a model monitor

    Parameters:

      name (str) – The new name to apply to the model monitor.

  delete()

    Deletes the specified Model Monitor and all its versions.

    Parameters:

      model_monitor_id (str) – Unique identifier of the Model Monitor to delete.

  list_monitor_alerts_for_monitor(realtime_monitor_id=None)

    Retrieves the list of monitor alerts for a specified monitor. One of the model_monitor_id or realtime_monitor_id is required but not both.

    Parameters:

      realtime_monitor_id (str) – The unique ID associated with the real-time monitor.

    Returns:

      A list of monitor alerts.

    Return type:

      list[MonitorAlert]