AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

AI-300 · LESSON 3

Design and implement a GenAIOps infrastructure20–25% · 3 topics

Learn the operating platform for generative AI: configuring Microsoft Foundry environments and platform, deploying and managing foundation models, and versioning prompts with source control.

3.1Implement Foundry environments and platform configuration

Concept

Microsoft Foundry (formerly Azure AI Foundry) is a generative AI platform handling models, agents, evaluation and monitoring together. Projects are created under a Foundry resource and separated by team or use. External services (Azure AI Search, Storage and so on) are connected securely through Connections.

How it works / steps

  1. Separate Foundry resources or projects for development, test and production
  2. Access connected services with managed identities and keyless authentication (Microsoft Entra ID)
  3. Close the network with private endpoints and by disabling public network access
  4. Minimize permissions with RBAC (Azure AI User, Azure AI Project Manager and so on)
  5. Make environments reproducible with IaC such as Bicep

Workplace example

The production project is usable only through private endpoints, and the app connects to Foundry and AI Search with a managed identity, using no API keys.

Common confusions and exam tips

Mini quiz

Q1. You want a Foundry project to access other Azure resources without keys. Which configuration is appropriate?

  1. Register a service principal secret in a connection
  2. Store access keys in the connection and manage them in Key Vault
  3. Create connections with an administrator’s Entra ID account and share them
  4. Use a managed identity and grant the needed roles with RBAC
Show answer and explanation

Answer: D Use a managed identity and grant the needed roles with RBAC

Managed identities and RBAC provide keyless, least-privilege access. [Other options] Key Vault still uses keys. Secrets require expiry and leak management. Sharing an administrator’s personal account is inappropriate.

Q2. You want to keep Foundry environments identical in development and production and make them reviewable. Which method is appropriate?

  1. Define and deploy the infrastructure with Bicep templates and Azure CLI
  2. Export ARM templates from the portal and fix them by hand each time
  3. Move the development resources as is into the production resource group
  4. Create production in the portal and visually compare settings with development
Show answer and explanation

Answer: A Define and deploy the infrastructure with Bicep templates and Azure CLI

Codifying configuration with IaC eliminates differences. [Other options] Moving removes the development environment. Visual comparison is not a reviewable definition. Hand-fixing exports lacks reproducibility.

Reference links

3.2Deploy and manage foundation models for production workloads

Concept

Foundry models differ in pricing, data processing location and capacity guarantees by deployment type: Standard (pay-as-you-go within a region), Global Standard (processed in datacenters worldwide with high quota), Data Zone (processed within zones such as US or EU), Provisioned (capacity reserved with PTU for stable latency) and Batch (asynchronous and cheaper).

How it works / steps

  1. Check data residency requirements and decide between Standard, Data Zone and Global
  2. Choose pay-as-you-go or Provisioned (PTU) based on traffic volume and stability requirements
  3. Check TPM (tokens per minute) quota and rate limits (429) and implement retries
  4. Manage model version auto-update settings and retirement schedules
  5. Use API Management as a gateway to centralize load balancing, token limits and logging

Workplace example

Chat handling EU customer data uses a Data Zone (EU) deployment, bulk overnight summarization uses Batch, and call center support with predictable peaks uses Provisioned.

Common confusions and exam tips

Mini quiz

Q1. In the trial phase, you want to start using a foundation model pay-as-you-go without managing infrastructure. Which deployment method is appropriate?

  1. Provisioned throughput (PTU)
  2. A Kubernetes online endpoint
  3. A serverless API endpoint
  4. Deployment to managed compute
Show answer and explanation

Answer: C A serverless API endpoint

Serverless API needs no infrastructure management and bills only for usage. [Other options] Managed compute and Kubernetes involve managing VMs or clusters and constant costs. PTU is a fixed cost that reserves capacity and does not suit trials.

Q2. Company-wide Q&A now needs high, stable throughput and predictable performance. What should you consider?

  1. A serverless API (pay-as-you-go) deployment
  2. A standard (pay-as-you-go) deployment with a throughput limit increase request
  3. A global batch deployment with high throughput limits
  4. Provisioned throughput (PTU)
Show answer and explanation

Answer: D Provisioned throughput (PTU)

PTU reserves capacity to get stable performance even under heavy load. [Other options] Global batch suits asynchronous jobs, not interactive Q&A. Pay-as-you-go is affected by contention and less predictable. A higher limit is still not reserved capacity.

Reference links

3.3Implement prompt versioning and management with source control

Concept

Prompts are like “code” that determines app behavior, so version them in Git and review and evaluate changes before release. Place them in the repository as Prompty files (a format combining prompt and model settings) or templates, combined with an evaluation pipeline.

How it works / steps

  1. Manage prompts and model parameters in the repository as files such as Prompty
  2. Submit changes as pull requests for review
  3. Run automated evaluation against an evaluation dataset in CI and fail if below criteria
  4. Tag passing versions and deploy them to production
  5. Record the prompt version used in production in traces

Workplace example

Even a one-line change to the system message is merged only after GitHub Actions measures groundedness and relevance on 200 evaluation items and confirms they do not drop below the previous version.

Common confusions and exam tips

Mini quiz

Q1. The model version used in production changed without notice and its output tendencies shifted. Which is an appropriate preventive measure?

  1. Enable automatic updates and learn about new version releases through notifications
  2. Explicitly manage the model version and roll out updates gradually after evaluation
  3. When output changes, fix the prompt to match the previous tendencies
  4. When a new version comes out, switch everything over without evaluation
Show answer and explanation

Answer: B Explicitly manage the model version and roll out updates gradually after evaluation

Pinning versions and gradual rollout after evaluation prevents unexpected changes. [Other options] Automatic updates cannot prevent unannounced changes. Fixing prompts after the fact is not prevention. Switching without evaluation causes the same problem.

Q2. You want to be sure to notice when a model’s retirement date approaches. Which practice is most appropriate?

  1. Consider switching to another model after errors appear past the retirement date
  2. Check model lifecycle information and notifications and plan the switch in advance
  3. With automatic updates enabled, every deployment can move to new versions without evaluation
  4. Keep redeploying the same model version to avoid retirement
Show answer and explanation

Answer: B Check model lifecycle information and notifications and plan the switch in advance

Models have retirement dates. Check published schedules and notifications and plan evaluation and switching ahead. [Other options] Waiting for errors stops operations. Redeploying does not change the retirement date. Even with automatic updates, evaluation and planning are needed, and it does not apply to every deployment.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)