AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

G検定 · LESSON 6

Toward social implementation of AI; mathematical statistics5%（当サイト設定） · 3 topics

Learn how to run AI projects, collecting, processing and analyzing data, and the math and statistics AI needs.

6.1Running AI projects

Running AI projects (CRISP-DM, CRISP-ML, PoC, MLOps, BPR, agile, Web APIs, cloud, Docker, Jupyter Notebook)

Concept

CRISP-DM is a standard process for data analysis projects, repeating business understanding → data understanding → data preparation → modeling → evaluation → deployment. CRISP-ML extends it for machine learning, adding quality assurance and operation and monitoring. Check feasibility small with a PoC (proof of concept), and keep development and operation cycling with MLOps. Rethinking the work itself through BPR (business process reengineering) also matters. Agile is common in development, Web APIs and cloud in delivery, Docker for reproducing environments and Jupyter Notebook for trial and error.

How it works / steps

  1. Decide the business problem and evaluation metrics first (business understanding)
  2. Verify small with a PoC before full development
  3. Include post-launch monitoring and retraining (MLOps) in the plan

Workplace example

In a demand forecasting PoC, check accuracy on the past year's data, review the ordering workflow (BPR), then integrate into production as a Web API.

Common confusions and exam tips

Mini quiz

Q1. You plan to use AI for demand forecasting. What is the most appropriate purpose of a PoC before full development?

  1. Check feasibility and impact on a small scale, and decide on full development against evaluation criteria
  2. Give several vendors the same task and pick the cheapest proposal
  3. Have target accuracy guaranteed by contract before collecting training data
  4. Build a system at production scale and finalize operating procedures
Show answer and explanation

Answer: A Check feasibility and impact on a small scale, and decide on full development against evaluation criteria

In a PoC, decide evaluation metrics and pass criteria in advance. Beware of repeating PoCs without ever reaching production. [Other options] Production-scale building exceeds a PoC. Price comparison is procurement. Guaranteeing accuracy before checking data is hard in AI development.

Q2. A production demand forecasting model's accuracy fell gradually over six months. Which response follows MLOps best?

  1. Have a data scientist rebuild the model locally once a year
  2. Freeze the model that first scored high and accept the drop as within tolerance
  3. Monitor inputs and accuracy, and keep running retraining and deployment when drift is detected
  4. Retrain a larger model once and set up no monitoring afterward
Show answer and explanation

Answer: C Monitor inputs and accuracy, and keep running retraining and deployment when drift is detected

In operation, accuracy drops from shifts in data distribution (data drift) or in the input-answer relationship (concept drift), so monitoring and retraining are needed. [Other options] Freezing or a single retrain can't handle future drift. A yearly local rebuild isn't continuous monitoring, retraining and deployment.

Reference links

6.2Collecting, processing, analyzing and training on data

Collecting, processing, analyzing and training on data (open datasets, annotation, corpora, data leakage)

Concept

Data is collected from open datasets (ImageNet, MNIST, COCO etc.), your own data, the web and so on. Annotation is labeling data with correct answers; aligning quality criteria is important. A corpus is a large body of text data. Data leakage is the problem where information not available at prediction time (future information or information directly tied to the answer) mixes into training, making evaluation unduly high.

How it works / steps

  1. Check terms of data use (licenses, personal information)
  2. Document annotation criteria and check variation between annotators
  3. Remove from features any information unavailable at prediction time

Workplace example

In predicting hospital stay length, a feature computed from the “discharge date” was used in training, making evaluation abnormally high (data leakage).

Common confusions and exam tips

Mini quiz

Q1. Which is the most appropriate point of care for data annotation?

  1. Label quality doesn't affect model accuracy, so prioritize only speed
  2. Document labeling criteria and manage quality while checking variation between annotators
  3. Have as many annotators as possible label freely by their own judgment, without criteria
  4. Keep using labels once assigned, even if errors are found
Show answer and explanation

Answer: B Document labeling criteria and manage quality while checking variation between annotators

Annotation quality drives model performance. Clarify criteria, cross-check with several people and check agreement rates.

Q2. Which best describes “data leakage”?

  1. An information security problem where training data leaks outside and personal data is exposed
  2. A problem where storage runs short and old data is deleted automatically
  3. A problem where training data is too small and model accuracy doesn't rise enough
  4. A problem where information unavailable at prediction time mixes into training, making evaluation unduly good
Show answer and explanation

Answer: D A problem where information unavailable at prediction time mixes into training, making evaluation unduly good

Example: including a “cancellation procedure date” field in churn prediction. Accuracy won't hold in production.

Reference links

6.3Math and statistics for AI

Math and statistics for AI (mean, median, mode; variance, standard deviation; correlation, spurious correlation; normal, binomial, Poisson distributions; conditional probability; maximum likelihood; least squares; hypothesis testing; Euclidean and Mahalanobis distance, cosine similarity)

Concept

Representative values include the mean, median (robust to outliers) and mode; spread is expressed by variance and standard deviation. The correlation coefficient is the strength of a linear relation between two variables (−1 to 1); correlation doesn't mean causation, and apparent correlation from a third factor is spurious correlation. Probability distributions include the normal, binomial (number of successes) and Poisson (number of occurrences in a fixed time). Conditional probability underlies Bayes' theorem. Maximum likelihood estimates parameters maximizing the probability of the data; least squares minimizes the sum of squared errors. Hypothesis testing judges by p-value whether the null hypothesis can be rejected. Distances include Euclidean distance, Mahalanobis distance (accounting for spread) and cosine similarity (closeness of direction).

How it works / steps

  1. Check the distribution and outliers before choosing a representative value
  2. When you find a correlation, consider whether it is causal or spurious
  3. Use cosine similarity for similarity between text vectors

Workplace example

Ice cream sales and drowning accidents correlate, but it's a spurious correlation both caused by temperature; ice cream doesn't cause accidents.

Common confusions and exam tips

Mini quiz

Q1. Data show that days with more ice cream sales also have more drowning accidents. Which interpretation is most appropriate?

  1. It may be spurious correlation, appearing correlated through a third factor, temperature
  2. A correlation of 0.9 or more lets you conclude causation
  3. The high correlation means ice cream sales cause drownings
  4. There is reverse causation: more drownings raise ice cream sales
Show answer and explanation

Answer: A It may be spurious correlation, appearing correlated through a third factor, temperature

Example: ice cream sales and drowning accidents correlate because both depend on temperature. Correlation with other variables' influence removed is the partial correlation coefficient. [Other options] Correlation isn't causation. Reverse causation has no basis. Correlation size alone can't show causation; check with partial correlation removing the third factor.

Q2. Which best describes maximum likelihood estimation?

  1. Taking the most frequent value in the data as its representative value
  2. Forming a hypothesis and judging by probability whether data lets you reject it
  3. Choosing a line's coefficients to minimize the sum of squared differences between predictions and actuals
  4. Estimating parameters so the probability of obtaining the observed data (the likelihood) is maximized
Show answer and explanation

Answer: D Estimating parameters so the probability of obtaining the observed data (the likelihood) is maximized

[Other options] Minimizing the sum of squares is least squares, the most frequent value is the mode, and rejecting hypotheses is hypothesis testing (null and alternative hypotheses).

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)