AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

PMLE · LESSON 3

Section 3: Scaling prototypes into ML models約21% · 3 topics

Learn building models with cost, complexity and latency in mind, model training, and training hardware and distributed training.

3.1Building models with requirements in mind

Building models with cost, complexity, latency and scalability in mind (ARIMA, DNN, LLM; AutoML, BigQuery ML, Pipelines; deployment strategy; interpretability requirements)

Concept

Choose models from requirements (accuracy, latency, cost, interpretability, scalability). Candidates are statistical models such as ARIMA for time series, DNNs for complex unstructured data, and LLMs for diverse language tasks. In order of least effort, the development options are BigQuery ML and AutoML, flexible custom training, and automation with Pipelines. If decisions must be explained, for example due to regulation, use interpretable models or Explainable AI.

How it works / steps

  1. Set priorities among requirements (accuracy, speed, cost, explanation)
  2. First set a baseline with a simple model or BigQuery ML/AutoML
  3. Move to custom DNNs or LLMs as needed
  4. If there are interpretability requirements, check model type and explanation methods

Workplace example

Because credit screening requires explanations, first build an interpretable boosted tree model and confirm Explainable AI can show feature contributions.

Common confusions and exam tips

Mini quiz

Q1. For a credit model where regulation strongly demands interpretability, which approach is most appropriate to consider first?

  1. Skip explanations and notify customers of predictions only
  2. Choose an interpretable model and show reasons with Explainable AI
  3. Have an LLM freely write reasons and send them to customers as is
  4. Choose the most accurate deep network and skip explaining reasons
Show answer and explanation

Answer: B Choose an interpretable model and show reasons with Explainable AI

Linear models and decision trees, or feature attributions.

Q2. In which situation should fine-tuning NOT be considered?

  1. Prompts have grown too long, and you want lower cost and latency
  2. You want answers to reflect inventory information that changes daily
  3. You want the model to consistently adopt your company's specialized style
  4. You want higher accuracy on a specialized classification task with many examples
Show answer and explanation

Answer: B You want answers to reflect inventory information that changes daily

Handle frequently changing facts with RAG or tools.

Reference links

3.2Model training

Model training (organizing data in Cloud Storage and BigQuery, ingestion, custom training, Kubeflow on GKE, AutoML, Tabular Workflows, handling training failures, hyperparameter tuning, fine-tuning foundation models)

Concept

Organize training data in Cloud Storage (files: TFRecord, images, etc.) or BigQuery (tables). Training methods include custom training with your own containers or scripts, Kubeflow on GKE, AutoML, and Tabular Workflows for tabular data. Search for the best settings with hyperparameter tuning (such as Bayesian optimization with Vizier), and investigate the causes of training failures (out of memory, data mismatches, divergence) in logs. Foundation models can be adjusted with supervised tuning and similar methods.

How it works / steps

  1. Match data format and storage location to the training method
  2. Use a custom container if prebuilt containers aren't enough
  3. Search with a hyperparameter tuning job
  4. Save checkpoints so you can resume after failures or interruptions

Workplace example

Custom-train image classification with a prebuilt PyTorch container, and search learning rate and batch size with a tuning job.

Common confusions and exam tips

Mini quiz

Q1. What is the most appropriate advantage of using Vizier for hyperparameter tuning?

  1. It automatically adds training data and relabels it
  2. It sets model parameters (weights) one by one by hand
  3. It finds good combinations in few trials with Bayesian optimization and the like
  4. It always brute-forces every combination, maximizing computation
Show answer and explanation

Answer: C It finds good combinations in few trials with Bayesian optimization and the like

Agent Platform hyperparameter tuning uses Vizier.

Q2. You will run a custom training job on hundreds of millions of rows in BigQuery. Which loading approach is best?

  1. Read directly and efficiently from the job, for example with the BigQuery Storage Read API
  2. Export results to Cloud Storage as a single CSV and read that file
  3. Move the data to Cloud SQL and read it over JDBC
  4. Page through rows with the BigQuery REST tabledata.list call inside the job
Show answer and explanation

Answer: A Read directly and efficiently from the job, for example with the BigQuery Storage Read API

You can also specify BigQuery as the dataset. [Other options] A single CSV is too large to read in parallel. tabledata.list is slow for bulk reads. Moving to Cloud SQL adds effort and cost.

Reference links

3.3Training hardware and distributed training

Training hardware (CPU, GPU, TPU; data-parallel and model-parallel distributed training)

Concept

CPUs are general-purpose for small models, GPUs do parallel computation for deep learning in general, and TPUs are accelerators developed by Google specialized for matrix operations, suited to models with many large matrix operations (TensorFlow, JAX, PyTorch/XLA). Distributed training includes data parallelism, which splits data to train the same model on several devices, and model parallelism, which splits a large model that doesn't fit on one device.

How it works / steps

  1. Choose hardware from model size and framework
  2. Use data parallelism when data is large and you want faster training
  3. Use model parallelism when the model doesn't fit in one device's memory
  4. Monitor utilization and avoid oversized hardware

Workplace example

When fine-tuning a large language model that doesn't fit on one GPU, split it across several GPUs with model parallelism.

Common confusions and exam tips

Mini quiz

Q1. Which best describes data-parallel distributed training?

  1. Each worker processes different data, and gradients are aggregated
  2. Training runs on one machine, and only inference is distributed
  3. Each worker trains a different model on the same data and the best is chosen
  4. Model layers are split and placed on different devices
Show answer and explanation

Answer: A Each worker processes different data, and gradients are aggregated

If the model fits on one device, data parallelism is the default.

Q2. You want to adapt a large language model to internal terminology on a limited GPU budget. Which is the main benefit of parameter-efficient tuning such as LoRA?

  1. It retrains all weights, so accuracy improves more easily
  2. The tuned model no longer needs the base weights at inference
  3. It specializes the model even without training data
  4. It trains a few added parameters, saving compute and storage
Show answer and explanation

Answer: D It trains a few added parameters, saving compute and storage

Only adapters (small weights) are trained. [Other options] Training still needs data. LoRA adapters are used together with the base weights. Retraining all weights is full fine-tuning.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)