Amazon Web Services (AWS), an Amazon.com company, announced the general availability of Amazon Augmented Artificial Intelligence (A2I), a fully managed service that makes it easy to add human review to machine learning predictions to improve model and application accuracy by continuously identifying and improving low confidence predictions. Amazon A2I helps developers add human review for model predictions to new or existing applications using reviewers from Mechanical Turk, third party vendors, or their own employees. Amazon A2I makes it easier for developers to build the human review system, structure the review process, and manage the human review workforce. For example, developers could use Amazon A2I to quickly spin up and manage a workforce of humans to review and validate the accuracy of machine learning predictions for an application that extracts financial information from scanned mortgage documents or an application that uses image recognition to identify counterfeit items online, so that the quality of results improve over time. There are no upfront commitments to use Amazon A2I, and users pay only for each review needed. To get started with Amazon A2I, visit aws.amazon.com/augmented-ai
Today, machine learning provides highly accurate predictions (known as “inferences”) for a variety of use cases, including identifying objects in images, extracting text from scanned documents, or transcribing and understanding spoken language. In each case, machine learning models provide an inference and a confidence score that expresses how certain the model is in its prediction. The higher the confidence number, the more the result can be trusted. Typically, when developers receive a high confidence result they can trust that the prediction is accurate, and, depending on the use case, they can use it to fully automate a process. For example, developers of a social media application that matches a user’s photos to celebrity faces might rely on an 80% confidence score to generate and return a lot of entertaining matches. However, there are other times when it is strongly recommended to have both high confidence (up to 99%) and human review, such as public safety use cases involving law enforcement. In situations where the confidence score is lower than desired and/or human judgment is required, reviews can be used to validate the prediction. This interplay between machine learning and human reviewers is critical to the success of machine learning systems, but human reviews are challenging and expensive to build and operate at scale, often involving multiple workflow steps, operating custom software to manage human review tasks and results, and recruiting and managing large groups of reviewers. As a result, developers sometimes spend more time managing the human review process than building the intended application, or they have to forego having human reviews, which leads to less confidence in deploying applications that utilize machine learning.
With Amazon A2I, developers can add human review to machine learning applications without the need to build or manage expensive and cumbersome systems for human review. Amazon A2I provides over 60 pre-built human review workflows for common machine learning tasks (e.g. object detection in images, transcription of speech, and content moderation, etc.) that allow machine learning predictions from Amazon Rekognition and Amazon Textract to be human-reviewed more easily. Developers who build custom machine learning models in Amazon SageMaker (or other on-premises or cloud tools) can set up human review for their specific use case in the Augmented AI console or via its Application Programming Interface (API). After setting a confidence threshold for model predictions, developers can choose to have predictions below that threshold reviewed by Amazon Mechanical Turk and its 500,000 global workforce of independent contractors, third-party organizations who specialize in business process outsourcing (e.g. iVision, CapeStart Inc., and iMerit), or their own private, in-house reviewers. Developers can specify the number of workers per review and Amazon A2I then routes each review to the precise number of reviewers. For example, a company building a system for processing financial loan applications using Amazon Textract can easily configure Amazon A2I to work with Amazon Textract outputs such that forms that have a confidence score less than 99% will be routed to human reviewers from their private workforce. Human-validated results are stored in Amazon Simple Storage Service (S3), and developers can set up Amazon CloudWatch Events notifications to review metadata about inference accuracy and retrieve the results.
“We often hear from our customers that Amazon SageMaker helps speed training, tuning, and deploying custom machine learning models, while fully managed services like Amazon Rekognition and Amazon Textract make it easy to build applications that incorporate machine learning without requiring any machine learning expertise. But even with these advancements, our customers still say there are critical use cases where human judgment is required like in law enforcement investigations, or times when human review can be used to resolve the ambiguity in predictions when confidence levels fall below a given threshold for less sensitive use cases, and the current human review process involves a lot of custom effort and cost,” said Swami Sivasubramanian, Vice President, Amazon Machine Learning, Amazon Web Services, Inc. “Today, we’re excited to help our customers remove another obstacle to building machine learning applications with the launch of Amazon A2I, which makes it significantly easier and faster to incorporate human judgment into machine learning applications in order to ensure higher quality predictions over a sustained period of time.”
Amazon A2I is available today in US East (N. Virginia), US East (Ohio), US West (Oregon), Canada (Central), EU West (London), EU West (Ireland), EU (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Seoul), and Asia Pacific (Mumbai).
