AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

PMLE · LESSON 2

Section 2: Collaborating within and across teams to manage data and models約16% · 3 topics

Learn data exploration and preprocessing, prototyping in notebooks, and experiment tracking.

2.1Data exploration and preprocessing

Data exploration and preprocessing (tabular, text, images; choosing among BigQuery, Dataflow, Spark and in-memory Python; Feature Store; protecting PII)

Concept

Choose the processing platform by data size and type. Use BigQuery for SQL processing of large tabular data, Dataflow (Apache Beam) for large batch and streaming transforms, Dataproc if you have Spark assets, and in-memory Python (pandas) is enough for small data. Manage features centrally in Feature Store and share them between online inference and offline training. Detect and mask PII with Sensitive Data Protection (formerly the DLP API).

How it works / steps

  1. Choose the platform from data volume and processing type (SQL, streaming, Spark)
  2. Register features shared by training and inference in Feature Store
  3. Detect PII and mask or pseudonymize it
  4. Check missing values, outliers and imbalance, and preprocess

Workplace example

Aggregate a click stream in real time with Dataflow and write it to Feature Store, so the recommendation model reads the latest features at inference.

Common confusions and exam tips

Mini quiz

Q1. What is the most appropriate advantage of using Feature Store?

  1. Features are reused, and the same values are used in training and serving
  2. It automatically rewrites the training loss function to the optimal one
  3. It automatically deletes all PII in training data to anonymize it
  4. It automatically converts notebook code into production pipelines
Show answer and explanation

Answer: A Features are reused, and the same values are used in training and serving

Use BigQuery as offline storage and fetch with low latency via online serving.

Q2. What is the most appropriate way to organize tabular, text and image data for training?

  1. Store all data only on the notebook's local disk
  2. Print text for storage and re-enter it by hand at training time
  3. Put tables in BigQuery and images and large files in Cloud Storage
  4. Embed all images as Base64 in BigQuery string columns
Show answer and explanation

Answer: C Put tables in BigQuery and images and large files in Cloud Storage

Choose storage that fits the format. Image paths and metadata are sometimes managed in BigQuery.

Reference links

2.2Prototyping in notebooks

Prototyping in notebooks (Agent Platform Workbench, Colab Enterprise, security and collaboration, PyTorch, scikit-learn, JAX, Model Garden models)

Concept

Agent Platform Workbench is a managed Jupyter environment, and Colab Enterprise is a managed notebook that makes collaboration easy. Both can be secured with IAM, VPC Service Controls, customer-managed encryption keys and more, and you can prototype with frameworks such as PyTorch, TensorFlow, scikit-learn and JAX, or with Model Garden models.

How it works / steps

  1. Choose Workbench or Colab Enterprise to fit how the team works
  2. Limit the data a notebook can access with service accounts and IAM
  3. Keep costs down with automatic idle shutdown
  4. Organize prototype code so it can move into a pipeline later

Workplace example

A data scientist reads BigQuery data in Colab Enterprise, prototypes a model with scikit-learn and shares the notebook with the team.

Common confusions and exam tips

Mini quiz

Q1. Which is the most appropriate security best practice for notebook environments?

  1. Assign a public IP so anyone can access it for collaboration
  2. Use least-privilege service accounts and protect with VPC Service Controls and CMEK
  3. Paste service account key files into notebooks and share them
  4. Have everyone share one admin-privileged account to use notebooks
Show answer and explanation

Answer: B Use least-privilege service accounts and protect with VPC Service Controls and CMEK

Avoid sharing key files; authenticate with user identities or least-privilege service accounts.

Q2. What is the most appropriate way to develop a PyTorch model in Workbench?

  1. PyTorch isn't supported, so always rewrite it as BigQuery ML SQL
  2. Notebooks can't train, so calculate on paper first and then enter results
  3. PyTorch models must always be built from the AutoML screen
  4. Use an image with PyTorch and attach a GPU if needed
Show answer and explanation

Answer: D Use an image with PyTorch and attach a GPU if needed

Workbench and Colab Enterprise support PyTorch, scikit-learn, JAX and more.

Reference links

2.3Experiment tracking

Experiment tracking (Experiments, Pipelines, Kubeflow Pipelines, evaluating predictive and generative AI, LLM-as-a-judge, lineage tracking with ML Metadata)

Concept

Agent Platform Experiments records and compares parameters, metrics and artifacts for each experiment run. Combined with Pipelines runs (Kubeflow Pipelines format), ML Metadata tracks the lineage of data, models and runs. For generative AI, measure quality with the evaluation service and LLM-as-a-judge evaluation, which uses an LLM as the grader.

How it works / steps

  1. Record parameters and metrics for each experiment
  2. Also record data and code versions so good results can be reproduced
  3. Track which data produced which model with ML Metadata
  4. Compare generative AI with an evaluation dataset and LLM-as-a-judge

Workplace example

Compare 20 runs with different learning rates and tree depths in Experiments, check the lineage of the best run's model in ML Metadata, and register it.

Common confusions and exam tips

Mini quiz

Q1. What is the most appropriate caution when using LLM-as-a-judge?

  1. The judge LLM can be biased, so calibrate it against human evaluation
  2. The judge LLM is always more accurate than humans, so human evaluation is unnecessary
  3. The judging LLM and the judged LLM must always be the same model
  4. Don't give criteria; let the judge LLM score freely
Show answer and explanation

Answer: A The judge LLM can be biased, so calibrate it against human evaluation

Make the criteria (rubric) clear and check agreement with human evaluation.

Q2. When is it most appropriate to use Agent Platform Pipelines as the experiment environment?

  1. You want staff to score model outputs one by one
  2. You want to run one line of code interactively right away
  3. You want to run everything from preprocessing to evaluation reproducibly
  4. You want to leave comments together in a notebook
Show answer and explanation

Answer: C You want to run everything from preprocessing to evaluation reproducibly

Pipelines are for reproducibility and automation; interactive trials are for notebooks.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)