Policies

Once you're in production, it's impossible to efficiently monitor multiple models and metrics by simply observing each model and segment to find potential issues. That's why Superwise provides a flexible monitoring policy builder that lets you define any issues you want the platform to scan for and monitor on an ongoing basis.

How does this work?

When you set up a monitoring policy, Superwise will analyze your data constantly to detect anomalies in your model's behavior. This is done based on the scope and conditions you set; you also configure the frequency and alerting mechanism.

Once an anomaly occurs (see Incidents), Superwise will alert you and will send you a direct link to the incident so you can investigate.

Configuring a policy consists of three parts:

1. Policy scope - Configure any logical combination to be monitored. What metric applied on which entities (features/predictions/labels) do you want to monitor? Across which models and segments?
2. Conditions and sensitivity - When should Superwise trigger an anomaly? Do you want Superwise to apply dynamic anomaly detection or use a fixed threshold? How sensitive should the threshold be?
3. Scheduling and notifications - Configure when to scan and how you should be notified upon violation.

Click here to understand how to configure a policy.

Investigation of incidents

Once an incident occurs, Superwise will open a new incident in the Incidents screen, as shown here. You can also read more about investigation capabilities.