AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

AI-103 · LESSON 3

Implement computer vision solutions10–15% · 3 topics

Learn image and video generation, multimodal understanding workflows and responsible AI for multimodal content.

3.1Design and implement image- and video-generation solutions

Concept

Foundry offers image generation and editing models (gpt-image family, DALL-E family and so on) and video generation models (such as Sora). Specify content, style and composition in the prompt; editing (inpainting) and generation from reference images are also possible. Generated content carries C2PA provenance information (content credentials).

How it works / steps

  1. Decide the purpose (ad concepts, product images, training videos) and the required resolution or length
  2. Write the subject, style, composition and what to avoid concretely in the prompt
  3. Revise existing images with editing or reference images
  4. Have people review output for rights or expression issues
  5. Indicate that content is generated and keep provenance information

Workplace example

Marketing generates 10 banner concepts for a new product and a designer picks one to finish. Instructions involving real people or logos are refused by filters.

Common confusions and exam tips

Mini quiz

Q1. You want to generate seasonal campaign images from a product photo (reference image) and text instructions. Which implementation is appropriate?

  1. Extract tags from the photo with image analysis and search for new images by tag
  2. Have a multimodal model describe the photo and generate an image from that text alone
  3. Use image generation that takes a text prompt and a reference image as input
  4. Extract features from the photo with Content Understanding and fit them into a template
Show answer and explanation

Answer: C Use image generation that takes a text prompt and a reference image as input

Image generation with a reference image creates new images while keeping the original product’s characteristics. [Other options] Searching by tags is not generation. A description alone cannot keep the product’s appearance. Content Understanding extracts and does not generate images.

Q2. You want to replace only part of the background of a generated image. Which technique is appropriate?

  1. Inpainting with a specified mask
  2. Raise the image size and quality settings at generation
  3. Change the reference image to create image variations
  4. Regenerate the whole image with the same prompt
Show answer and explanation

Answer: A Inpainting with a specified mask

Inpainting edits only the masked area. [Other options] Full regeneration or variations cannot replace only the specified part. Size and quality settings do not specify the edit area.

Reference links

3.2Design and implement multimodal understanding workflows

Concept

Multimodal models (such as the GPT-4o family) can take images and text together and understand and describe them. Azure Content Understanding is a Foundry Tools service that extracts information from documents, images, audio and video according to a defined schema (fields). Azure AI Vision provides routine image analysis such as OCR, tagging and captions.

How it works / steps

  1. Decide the input type (image, video, document) and the desired output (description, field extraction, classification)
  2. For free-form description or Q&A, pass the image to a multimodal model
  3. For extracting fixed fields, define a schema in a Content Understanding analyzer
  4. Use Azure AI Vision for routine OCR or tagging
  5. Validate output and have people check low-confidence results

Workplace example

Pass insurance accident photos and descriptions to a multimodal model to summarize the damage, and extract amounts and parts from repair quotes with Content Understanding.

Common confusions and exam tips

Mini quiz

Q1. You want to add alt text that meets accessibility requirements to images on a website. Which method is appropriate?

  1. Attach the same description template to all images and set it in bulk per page
  2. Have a multimodal model describe each image and adjust the length to the requirements
  3. Set the text read by OCR from the image as the alt text as is
  4. Use keywords extracted by image analysis tagging, comma-separated, as alt text
Show answer and explanation

Answer: B Have a multimodal model describe each image and adjust the length to the requirements

Multimodal models can generate descriptions that match the image; adjust length for the purpose. [Other options] Tag lists or OCR text do not describe the image’s content or purpose. One template cannot convey each image’s content.

Q2. You want to show a photo of equipment, ask “Is this part damaged?” and get an answer grounded in the image. Which option is appropriate?

  1. Document Intelligence layout analysis and table extraction
  2. Similar-image search using image embeddings
  3. Object detection and tagging with image analysis
  4. Question answering grounded in the image with a multimodal model
Show answer and explanation

Answer: D Question answering grounded in the image with a multimodal model

Implement visually grounded QA with a multimodal model that accepts images as input. [Other options] Object detection and tagging do not judge damage in response to a question. Layout analysis is for documents, and similar-image search is not question answering.

Reference links

3.3Implement responsible AI for multimodal content

Concept

For AI that handles images and video, it is important to address both generation and input of harmful images, handle sensitive information such as faces, and show the provenance of generated content. Combine content filters (image input and output), Azure AI Content Safety image moderation, applications for limited-access features (such as face recognition) and C2PA provenance information.

How it works / steps

  1. Configure filters on both image input and output
  2. Screen user-uploaded images with Content Safety
  3. For limited-access features such as face identification, check the usage requirements and application
  4. Indicate AI generation on output and keep provenance information
  5. Regularly check false positives and misses and adjust thresholds

Workplace example

On an image board where users post images, Content Safety automatically holds violent or sexual images, and staff decide borderline cases.

Common confusions and exam tips

Mini quiz

Q1. You want to classify unsafe or prohibited image content automatically. Which option is appropriate?

  1. Generating alt text from images
  2. Searching for similarity to prohibited images with image embeddings
  3. Safety filters for multimodal content
  4. Image analysis tagging and object detection
Show answer and explanation

Answer: C Safety filters for multimodal content

Apply safety filters to visual content to classify and block it. [Other options] Tagging and alt text describe content but do not classify safety. Similarity search cannot determine categories of prohibited content.

Q2. You want to automatically determine whether images uploaded by users contain inappropriate content. Which option is most appropriate?

  1. Generate an image description and pass the text through a text filter
  2. Use image content safety assessment (categories and severity)
  3. Check whether image analysis tags contain prohibited words
  4. Accept or reject by the image’s file name and extension
Show answer and explanation

Answer: B Use image content safety assessment (categories and severity)

Image content safety assessment judges categories such as violence and sexual content with severity, and thresholds decide handling. [Other options] Tags and descriptions are unsuited to judging inappropriateness and miss things. File names cannot judge content.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)