Once you're in production, it's impossible to efficiently monitor multiple models and metrics by simply observing each model and segment to find potential issues. That's why Superwise provides a flexible monitoring policy builder that lets you define any issues you want the platform to scan for and monitor on an ongoing basis.
When you set up a monitoring policy, Superwise will analyze your data constantly to detect anomalies in your model's behavior. This is done based on the scope and conditions you set; you also configure the frequency and alerting mechanism.
Once an anomaly occurs (see Incidents), Superwise will alert you and will send you a direct link to the incident so you can investigate.
- Policy scope - Configure any logical combination to be monitored. What metric applied on which entities (features/predictions/labels) do you want to monitor? Across which models and segments?
- Conditions and sensitivity - When should Superwise trigger an anomaly? Do you want Superwise to apply dynamic anomaly detection or use a fixed threshold? How sensitive should the threshold be?
- Scheduling and notifications - Configure when to scan and how you should be notified upon violation.
Click here to understand how to configure a policy.
Once an incident occurs, Superwise will open a new incident in the Incidents screen, as shown here. You can also read more about investigation capabilities.
Updated 7 months ago