AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

G検定 · LESSON 5

Applications of deep learning16%（当サイト設定） · 7 topics

Learn image recognition, natural language processing, speech processing, deep reinforcement learning, data generation, transfer learning and multimodal AI, and interpretability and compression.

5.1Image recognition

Image recognition (object recognition, detection, segmentation, pose estimation; AlexNet, VGG, GoogLeNet, ResNet, EfficientNet, R-CNN family, YOLO, SSD, U-Net, Mask R-CNN, Vision Transformer)

Concept

Image recognition tasks include object recognition (the class of the whole image), object detection (object locations as rectangles), segmentation (a class per pixel: semantic, instance, panoptic) and pose estimation (human joint positions, e.g. OpenPose). Recognition models evolved AlexNet → VGG (stacked 3×3 convolutions) → GoogLeNet (Inception modules) → ResNet (skip connections) → EfficientNet (balanced scaling of depth, width and resolution). Detection has two-stage R-CNN models (Faster R-CNN) and one-stage YOLO and SSD; segmentation has U-Net and Mask R-CNN; Vision Transformer splits images into patches for a Transformer.

How it works / steps

  1. Decide the task from the needed output (class, rectangle, pixel, joint)
  2. For detection where speed matters, choose one-stage YOLO or SSD
  3. U-Net is common for medical image segmentation

Workplace example

On a factory line, use YOLO to detect part positions in real time, combined with a segmentation model to show scratch regions pixel by pixel.

Common confusions and exam tips

Mini quiz

Q1. You are choosing an image recognition task. In autonomous driving, you want to assign a class such as “road,” “sidewalk” or “sky” to each pixel. Overlapping cars need not be told apart. Which task is this?

  1. Object recognition (image classification)
  2. Instance segmentation
  3. Semantic segmentation
  4. Object detection
Show answer and explanation

Answer: C Semantic segmentation

Semantic segmentation classifies per pixel and doesn't separate instances of the same class. (Deciding point: per-pixel class, no instances.)

Q2. You are choosing an image recognition task. You want to separate several overlapping people into individual regions at the pixel level. Which task is this?

  1. Object recognition (image classification)
  2. Instance segmentation
  3. Semantic segmentation
  4. Object detection
Show answer and explanation

Answer: B Instance segmentation

Instance segmentation (Mask R-CNN etc.) separates regions per instance. Combining both is panoptic segmentation. (Deciding point: per-instance regions, per pixel.)

Reference links

5.2Natural language processing

Natural language processing (BoW, TF-IDF, word2vec, fastText, ELMo, BERT, GPT, LLMs, GLUE)

Concept

BoW (Bag of Words) represents documents by word counts; TF-IDF expresses word importance by frequency in the document and rarity across documents. word2vec turns words into distributed representations (vectors), with CBOW, which predicts the center word from surrounding words, and skip-gram, the reverse. fastText also uses substrings and is strong on unknown words; ELMo gets context-dependent representations with bidirectional LSTMs. BERT is a Transformer encoder pretrained with masked word prediction (MLM) and next sentence prediction. GPT is a Transformer decoder predicting the next word, which scaled up into LLMs. GLUE is a benchmark measuring language understanding performance.

How it works / steps

  1. Organize the evolution of word representations (counting → distributed → context-dependent)
  2. Understand the difference between BERT (understanding) and GPT (generation)
  3. Fine-tune pretrained models to suit the use

Workplace example

Fine-tune BERT to classify inquiries, and use a GPT-type LLM to generate FAQ answers.

Common confusions and exam tips

Mini quiz

Q1. You are choosing a representation of documents or words in NLP. You want to learn, from surrounding words, dense vectors (distributed representations) where words close in meaning are close, as in “king − man + woman ≈ queen.” Which fits best?

  1. n-gram
  2. word2vec
  3. TF-IDF
  4. BoW (Bag-of-Words)
Show answer and explanation

Answer: B word2vec

word2vec has CBOW and skip-gram. (Deciding point: learning dense vectors that capture meaning.)

Q2. You are choosing a representation of documents or words in NLP. You want to count the probability of word sequences using runs of N consecutive characters or words as units. Which fits best?

  1. TF-IDF
  2. BoW (Bag-of-Words)
  3. n-gram
  4. word2vec
Show answer and explanation

Answer: C n-gram

An n-gram is a unit of N consecutive items, long used in language models and search indexes. (Deciding point: runs of N consecutive items.)

Reference links

5.3Speech processing

Speech processing (A-D conversion, PCM, FFT, MFCC, hidden Markov models, WaveNet, CTC, speaker identification)

Concept

Speech captured by a microphone as an analog signal is digitized by A-D conversion (sampling, quantization) (PCM). The fast Fourier transform (FFT) splits it into frequency components, and MFCC (mel-frequency cepstral coefficients), a feature reflecting human hearing, is used. Conventional speech recognition modeled phoneme sequences with hidden Markov models (HMM). WaveNet generates speech waveforms directly and greatly improved synthesis quality. CTC learns alignments between input and output sequences of different lengths. Speaker identification determines who is speaking.

How it works / steps

  1. Understand the digitization flow (sampling → quantization → encoding)
  2. Understand the meaning of features (spectrum, MFCC)
  3. Distinguish technologies used in speech recognition from those in speech synthesis

Workplace example

Transcribe call center recordings with CTC-based speech recognition, and separate operator and customer utterances with speaker identification.

Common confusions and exam tips

Mini quiz

Q1. You are choosing a speech processing technique. Which DeepMind model generated speech waveforms directly by predicting the next sample from past ones, achieving natural speech synthesis?

  1. CTC
  2. Hidden Markov model
  3. WaveNet
  4. MFCC (mel-frequency cepstral coefficients)
Show answer and explanation

Answer: C WaveNet

WaveNet uses dilated causal convolution and greatly raised speech synthesis quality. (Deciding point: generating waveforms directly.)

Q2. You are choosing a speech processing technique. Which method, in speech recognition before deep learning spread, was used to model phoneme state transitions with probabilities?

  1. Hidden Markov model
  2. MFCC (mel-frequency cepstral coefficients)
  3. CTC
  4. WaveNet
Show answer and explanation

Answer: A Hidden Markov model

The hidden Markov model (HMM) was long used as the acoustic model. (Deciding point: a probabilistic model of state transitions.)

Reference links

5.4Deep reinforcement learning

Deep reinforcement learning (DQN, Double DQN, Rainbow, A3C, PPO, AlphaGo/AlphaStar, RLHF, sim2real, offline RL)

Concept

DQN approximates Q-learning's action values with deep learning, stabilizing training with experience replay (randomly reusing past experience) and a target network, and achieved results on Atari games. Double DQN curbs Q-value overestimation; Rainbow combines several improvements. A3C learns asynchronously with multiple agents; PPO stabilizes by limiting the size of policy updates. AlphaGo (Go) and AlphaStar (StarCraft II) are representative achievements. RLHF tunes LLMs with human ratings. sim2real moves policies learned in simulation to the real world; offline RL learns only from collected data.

How it works / steps

  1. Understand DQN's stabilizing tricks (experience replay, target network)
  2. When real-world trials are dangerous or costly, use simulation and sim2real
  3. If you can't interact with the environment, consider offline RL

Workplace example

Learn robot arm grasping in simulation, randomly varying lighting and friction (domain randomization) to ease the move to the real machine (sim2real).

Common confusions and exam tips

Mini quiz

Q1. You want to align a chat AI with human preferences and safety. Which method has people compare outputs, builds a reward model from that, and tunes with reinforcement learning?

  1. Instruction tuning
  2. Knowledge distillation
  3. RLHF
  4. DPO
Show answer and explanation

Answer: C RLHF

RLHF aligns LLM outputs with human preferences and safety. It trains a reward model and optimizes with PPO or similar. [Other options] Instruction tuning is supervised learning on instruction–answer pairs. Distillation transfers a large model's knowledge. DPO optimizes directly on preference data without a reward model or RL.

Q2. Which DeepMind agent reached top-player level in StarCraft II?

  1. AlphaStar
  2. OpenAI Five
  3. AlphaGo Zero
  4. Agent57
Show answer and explanation

Answer: A AlphaStar

AlphaStar also used multi-agent reinforcement learning. [Other options] Agent57 played 57 Atari games, AlphaGo Zero learned Go by self-play, and OpenAI Five played Dota 2.

Reference links

5.5Data generation

Data generation (GAN, DCGAN, Pix2Pix, CycleGAN, diffusion models, NeRF)

Concept

A GAN (generative adversarial network) trains a generator making realistic data against a discriminator telling real from fake. DCGAN is a GAN using convolutions; Pix2Pix learns image translation from paired images (line art and photos etc.); CycleGAN learns translation between two unpaired domains (horses and zebras etc.). Diffusion models learn to reverse a noise-adding process and generate images from noise. NeRF represents 3D space from photos taken from multiple viewpoints and generates images from new viewpoints.

How it works / steps

  1. Use Pix2Pix with paired data, CycleGAN without
  2. Use diffusion models for high-quality, diverse image generation
  3. Use NeRF to reconstruct 3D scenes

Workplace example

To convert daytime city photos into a nighttime mood from (unpaired) day and night photos, use CycleGAN.

Common confusions and exam tips

Mini quiz

Q1. Which best describes a GAN (generative adversarial network)?

  1. It generates images by reversing a noise-adding process, removing noise step by step
  2. It learns to compress input to a latent representation and restore it, detecting anomalies by reconstruction error
  3. It generates text by repeatedly predicting the next word from the previous ones
  4. It trains a generator and a discriminator against each other to generate realistic data
Show answer and explanation

Answer: D It trains a generator and a discriminator against each other to generate realistic data

GANs were proposed by Ian Goodfellow and colleagues. DCGAN, using CNNs, stabilized image generation.

Q2. You are choosing a data generation model. When you can prepare many corresponding image pairs such as line art and colored versions, you want to learn conversion from line art to colored images. Which fits best?

  1. Pix2Pix
  2. Diffusion model
  3. NeRF
  4. CycleGAN
Show answer and explanation

Answer: A Pix2Pix

Pix2Pix learns paired image translation with a conditional GAN.

Reference links

5.6Transfer learning and multimodal AI

Transfer learning and fine-tuning (few-shot, catastrophic forgetting, self-supervised learning) and multimodal AI (CLIP, DALL-E, foundation models, zero-shot)

Concept

Transfer learning uses a trained model for another task; fine-tuning uses trained weights as initial values and trains further on new data. Few-shot performs a task with a few examples; zero-shot with none. Catastrophic forgetting is losing earlier knowledge when learning a new task. Self-supervised learning pretrains without labels on problems made from the data itself (such as predicting masked parts). CLIP maps images and text into the same space; DALL-E generates images from text.

How it works / steps

  1. With little data, use transfer learning from a trained model
  2. Use large unlabeled data through self-supervised learning
  3. For linking images and text, use models like CLIP

Workplace example

For product image search, use CLIP to find images close to the text “red sneakers” without extra training (zero-shot).

Common confusions and exam tips

Mini quiz

Q1. Which best describes “catastrophic forgetting”?

  1. Model accuracy gradually falling due to label errors in the training data
  2. Performance on previously learned tasks dropping sharply after learning a new task
  3. Prediction accuracy falling over time as data trends change in operation
  4. Error suddenly growing and diverging mid-training because the learning rate is too large
Show answer and explanation

Answer: B Performance on previously learned tasks dropping sharply after learning a new task

It tends to occur in fine-tuning. [Other options] Accuracy falling in operation is data drift or concept drift.

Q2. What do you call a large model pretrained on vast data that adapts to many tasks such as translation, summarization and classification with extra tuning?

  1. Multi-task learning
  2. Expert system
  3. Transfer learning
  4. Foundation model
Show answer and explanation

Answer: D Foundation model

GPT, BERT and CLIP are examples of foundation models. The idea relates to multi-task learning, where one model learns several tasks. [Other options] Multi-task learning trains several tasks at once. Transfer learning reuses learned knowledge for another task and isn't a model name. Expert systems run on rules.

Reference links

5.7Model interpretability and compression

Model interpretability (XAI, CAM, Grad-CAM, LIME, SHAP, Permutation Importance) and model compression (distillation, pruning, quantization, lottery ticket hypothesis, edge AI)

Concept

XAI (explainable AI) shows the reasons for AI decisions. CAM and Grad-CAM show where in an image the model focused with a heatmap; LIME explains by approximating with a simple model around a prediction; SHAP shows each feature's contribution with Shapley values from game theory. Permutation Importance measures importance by the performance drop when a feature is shuffled. Compression includes distillation (training a small student model on a large teacher model's outputs), pruning (removing unneeded weights) and quantization (lowering weight precision). The lottery ticket hypothesis holds that a large network contains a small subnetwork that performs well alone. Edge AI runs inference on the device.

How it works / steps

  1. Choose a method by what's being explained (image focus areas or feature contributions)
  2. To run on devices, compress with quantization, pruning and distillation
  3. Check the accuracy drop after compression

Workplace example

To run image classification in a smartphone app, quantize the model to 8 bits and use Grad-CAM to check the focus areas behind misclassifications.

Common confusions and exam tips

Mini quiz

Q1. Which best describes A-D conversion for handling speech on computers?

  1. Judging who is speaking from voice features
  2. Creating human-like speech waveforms from a character string
  3. Decomposing the waveform into frequency components to find which pitches it contains
  4. Taking the analog waveform at fixed intervals (sampling) and replacing it with numbers (quantization)
Show answer and explanation

Answer: D Taking the analog waveform at fixed intervals (sampling) and replacing it with numbers (quantization)

PCM is a typical format of digital audio from A-D conversion. Frequency decomposition is the FFT, judging the speaker is speaker identification, and the second option is speech synthesis.

Q2. Which is the most fitting situation where explainable AI (XAI) is especially needed?

  1. Sorting many images by capture date regardless of user preferences
  2. Showing meeting room availability on a calendar
  3. Situations needing reasons for decisions, such as loan screening or medical diagnosis support
  4. Randomly varying game character movements to make play more fun
Show answer and explanation

Answer: C Situations needing reasons for decisions, such as loan screening or medical diagnosis support

For high-impact decisions, interpretability is needed to explain the basis, find errors and check fairness.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)