AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

MLA-C02 · LESSON 3

Deployment and Orchestration of ML and AI Workflows24% · 3 topics

Learn deployment infrastructure for ML and AI models, provisioning and configuring resources, and MLOps automation and CI/CD.

3.1Manage deployment infrastructure

3.1 Manage deployment infrastructure for ML and AI model types.

Concept

SageMaker inference options are real-time inference (low latency, always on), serverless inference (intermittent traffic, with cold starts), asynchronous inference (large payloads, long processing, queued requests, scales to 0) and batch transform (bulk processing of large data). Consolidate many small models with multi-model endpoints and multiple containers with multi-container endpoints.

How it works / steps

  1. Check latency, payload size, processing time and traffic patterns
  2. Real-time for low-latency constant use; serverless for intermittent use
  3. Asynchronous inference for large inputs or processing that takes minutes
  4. Batch transform for periodic bulk predictions
  5. Consolidate many per-customer models with a multi-model endpoint

Workplace example

Consolidate 1,000 per-customer demand forecasting models into one multi-model endpoint, loading only invoked models into memory to lower cost.

Common confusions and exam tips

Mini quiz

Q1. You will run your own inference container on a SageMaker AI endpoint. Where is the most appropriate place to store the container?

  1. Push the image to an Amazon ECR repository
  2. Put only the Dockerfile in an Amazon S3 bucket
  3. Put the source code on an Amazon EFS file system
  4. Keep it in a local registry on a staff PC
Show answer and explanation

Answer: A Push the image to an Amazon ECR repository

SageMaker AI pulls container images from ECR.

Q2. Which statement about SageMaker serverless inference is incorrect?

  1. Capacity goes to 0 when unused, and you are charged only for usage
  2. You configure memory size and maximum concurrency
  3. You can specify GPU instances to run large deep learning models
  4. After a period of no use, the first request may experience a cold start
Show answer and explanation

Answer: C You can specify GPU instances to run large deep learning models

Serverless inference does not support GPUs. If you need GPUs, use real-time or asynchronous inference.

Reference links

3.2Provision and configure resources

3.2 Provision and configure resources for ML and AI workloads based on existing architecture and requirements.

Concept

Configure endpoint instance types (CPU, GPU, Inferentia) and sizes, auto scaling (target tracking on InvocationsPerInstance and others) and placement inside a VPC according to requirements. SageMaker Inference Recommender recommends the best instances through load testing. Define infrastructure with CloudFormation or CDK; Bedrock also supports importing your own models (Custom Model Import).

How it works / steps

  1. Compare candidate instances' performance and cost with Inference Recommender
  2. Configure the auto scaling policy and minimum and maximum instance counts
  3. Place endpoints inside a VPC and allow only necessary traffic
  4. Define endpoint configuration with IaC
  5. For models trained elsewhere, also consider Bedrock Custom Model Import

Workplace example

For an image classification endpoint, choose an Inferentia instance cheaper than GPU based on Inference Recommender results, and scale on invocations per instance.

Common confusions and exam tips

Mini quiz

Q1. On a real-time endpoint, requests for large documents that take 3 minutes to process fail with timeouts. Responses need not be returned immediately. What is the most appropriate next step?

  1. Move to a serverless inference endpoint and add memory
  2. Start a Batch Transform job for each request
  3. Use a larger instance to fit processing within the limit
  4. Move to an asynchronous inference endpoint, output to S3 and notify on completion
Show answer and explanation

Answer: D Move to an asynchronous inference endpoint, output to S3 and notify on completion

Real-time inference has a response time limit. For long processing, use asynchronous inference, which can process for up to 1 hour. [Other options] Serverless inference also has a response time limit. A larger instance isn't guaranteed to fit. Per-request Batch Transform starts slowly and isn't for individual requests.

Q2. Before releasing a new model to production, you want to find which instance types can meet latency and cost requirements. Which is the most appropriate method?

  1. Check training resource use in SageMaker Debugger and pick the same type
  2. Run load tests on candidate instances with SageMaker Inference Recommender
  3. Choose the inference instance type from Compute Optimizer recommendations
  4. Monitor and compare production latency metrics with SageMaker Model Monitor
Show answer and explanation

Answer: B Run load tests on candidate instances with SageMaker Inference Recommender

Inference Recommender load-tests candidate instances and compares latency, throughput and cost. [Other options] Model Monitor watches after release. Compute Optimizer doesn't load test SageMaker endpoints. Training resource use doesn't show inference performance.

Reference links

3.3Implement MLOps automated orchestration and CI/CD

3.3 Implement automated orchestration and continuous integration and continuous delivery (CI/CD) pipelines for MLOps and AI workloads.

Concept

SageMaker Pipelines automates the flow of data processing → training → evaluation → condition (meets the bar?) → model registration. Deploy models approved in Model Registry from staging to production with CodePipeline and similar. Switch safely with deployment guardrails (blue/green: all at once, canary, linear, with automatic rollback on alarms).

How it works / steps

  1. Define processing, training, evaluation and registration steps in Pipelines
  2. Register the model only when evaluation results meet the bar (condition step)
  3. Trigger the deployment pipeline on Model Registry approval
  4. Configure canary traffic shifting and automatic rollback with deployment guardrails
  5. Start pipelines on data arrival or on a schedule

Workplace example

When new data arrives in S3, EventBridge starts Pipelines; if AUC is 0.85 or higher the model is registered, and after approval CodePipeline rolls it out to production via canary.

Common confusions and exam tips

Mini quiz

Q1. You want to automatically roll back to the previous model if the error rate rises midway through updating an endpoint's model. Which is the most appropriate method?

  1. Specify CloudWatch alarms in deployment guardrails and configure automatic rollback
  2. Delete and recreate the endpoint for every update, and recreate it again if it fails
  3. Have staff watch the screen 24 hours a day and roll back to the previous model by hand if errors increase
  4. Disable error-rate alarms so staff are not notified during model updates
Show answer and explanation

Answer: A Specify CloudWatch alarms in deployment guardrails and configure automatic rollback

Deployment guardrails automatically roll back to the previous fleet when the specified CloudWatch alarms fire.

Q2. When Model Monitor finds a data drift violation, you want to start a retraining pipeline automatically. Which is the most appropriate configuration?

  1. Notify staff of violations via SNS and have them start the pipeline
  2. Catch the violation alarm with EventBridge and start a SageMaker Pipelines run
  3. Run the Model Monitor schedule hourly to detect violations sooner
  4. Add SageMaker Clarify bias monitoring to investigate the drift cause
Show answer and explanation

Answer: B Catch the violation alarm with EventBridge and start a SageMaker Pipelines run

EventBridge rules can start SageMaker Pipelines triggered by alarms or schedules. [Other options] More frequent monitoring doesn't start retraining. SNS leaves a manual step. Clarify adds detection and doesn't trigger retraining.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)