A machine learning service is a product

We believe that an ML service should be developed and treated as a product, meaning that you should apply the same behaviours and standards as you would when developing any other software product. These behaviours include:

• Identify, profile and maintain an active relationship with the end-users of your ML service. Work with your users to identify requirements that feed into your development backlog, involve your users in validation of features and improvements, notify them of updates and outages, and in general, work to keep your users happy.

• Maintain a roadmap of features and improvements. Continue to improve your service throughout its lifetime.

• Provide good user documentation.

• Actively test your service.

• Capture the iterations of your service as versions and help users migrate to newer versions. Clearly define how long you will support versions of your service, and whether you will run old and new versions concurrently.

• Understand how you will retire your service, or support users if you choose not to actively maintain it any longer.

• Have an operability strategy for your service. Build in telemetry that is exposed through monitoring and alerting tools, so you know when things go wrong. Use this data to gain an understanding of how your users actually use your service.

• Define who is supporting your service and provide runbooks that help support recovery from outages.

• Provide a mechanism for users to submit bugs and unexpected results, and work toward providing fixes for these in future releases.