AIAI Agent Dev Labaiagentdev-lab.com
Sign in to save progressSign in
Menu

AIP-C01 · LESSON 4

Operational Efficiency and Optimization for GenAI Applications12% · 3 topics

Learn cost optimization and resource efficiency, application performance optimization, and monitoring generative AI apps.

4.1Cost optimization and resource efficiency

4.1 Implement cost optimization and resource efficiency strategies.

Concept

Generative AI cost is driven mainly by token volume. Lower it with prompt caching (caching long, repeatedly used context to cut input cost and latency), choosing the right model (smaller models, distillation), prompt routing, batch inference, limiting output tokens and tuning the number of RAG results.

How it works / steps

  1. Measure token usage per model and feature
  2. Use prompt caching for long system prompts and document context
  3. Choose the smallest model that maintains quality (consider distillation too)
  4. Use batch inference for work that does not need immediacy
  5. Cut input tokens by reducing result counts and conversation history

Workplace example

In a chat that put the same 20-page product manual in context every time, use prompt caching to greatly reduce input cost and time to first response.

Common confusions and exam tips

Mini quiz

Q1. In long chat conversations, the whole history is sent every time, so input tokens keep growing. Which countermeasure is most appropriate?

  1. Send the whole history twice so the model does not forget
  2. Summarize older exchanges and send only the summary plus recent exchanges
  3. Send no history, only each question, ignoring earlier conversation
  4. Assume raising the max token count lowers input token cost
Show answer and explanation

Answer: B Summarize older exchanges and send only the summary plus recent exchanges

Sending only a summary of older exchanges plus recent ones keeps context while reducing tokens.

Q2. Answers have long preambles and much repetition, inflating output token cost. Which countermeasure is most appropriate?

  1. Ignore output length and show users the whole generated answer as is
  2. Assume a longer input prompt naturally makes output shorter
  3. Raise temperature so answers have different lengths each time
  4. Specify output format and length in the prompt and also set a cap with maxTokens
Show answer and explanation

Answer: D Specify output format and length in the prompt and also set a cap with maxTokens

Instructions on output format and length plus a maxTokens cap reduce unnecessary output tokens.

Reference links

4.2Optimize application performance

4.2 Optimize application performance.

Concept

Performance metrics are time to first token (TTFT), overall latency and throughput. Improve them with streaming, latency-optimized inference (using faster infrastructure on supported models), prompt caching, smaller models, parallel processing and shorter output. Secure throughput with cross-Region inference or Provisioned Throughput.

How it works / steps

  1. Set TTFT and latency targets
  2. Raise perceived speed with streaming
  3. Use latency-optimized inference on supported models
  4. Run independent processing (such as retrieval and classification) in parallel
  5. Use cross-Region inference or Provisioned Throughput for insufficient throughput

Workplace example

In a voice assistant, use latency-optimized inference and streaming to deliver the first response within 500 milliseconds.

Common confusions and exam tips

Mini quiz

Q1. Which statement about prompt caching is incorrect?

  1. It can reduce input token cost and latency
  2. The cache has a time to live (TTL)
  3. Even if the cached part's content changes every time, it reduces cost just as well
  4. It reduces processing of a long leading part that is sent repeatedly
Show answer and explanation

Answer: C Even if the cached part's content changes every time, it reduces cost just as well

Caching works when the same leading part repeats. If the content changes, the cache is not used.

Q2. Which statement about operating a vector store is incorrect?

  1. Even after changing the embedding model, old and new vectors can be compared as is
  2. Approximate search parameters such as HNSW let you balance recall and latency
  3. Adding replicas spreads read processing
  4. Quantization reduces vector memory
Show answer and explanation

Answer: A Even after changing the embedding model, old and new vectors can be compared as is

Vectors from different embedding models are not in the same space and cannot be compared.

Reference links

4.3Monitoring generative AI apps

4.3 Implement monitoring systems for GenAI applications.

Concept

CloudWatch offers Bedrock metrics (invocations, latency, token counts, throttling), generative AI observability (model invocation and agent traces) and AgentCore Observability (traces of agent sessions and tool calls). For quality monitoring, combine these with regular evaluation of samples from invocation logs.

How it works / steps

  1. Build dashboards and alarms for token counts, latency, errors and throttling
  2. Identify slow tools and failing steps in agent traces
  3. Monitor the number of guardrail interventions
  4. Regularly evaluate log samples to detect quality degradation
  5. Detect cost anomalies with Cost Anomaly Detection

Workplace example

An alarm detects that agent responses have slowed, and AgentCore traces show the inventory API tool is taking 6 seconds.

Common confusions and exam tips

Mini quiz

Q1. You want an early alert when the app's monthly cost looks likely to exceed the budget. Which method is most appropriate?

  1. Check the Bedrock cost trend weekly in Cost Explorer
  2. Create a budget in AWS Budgets and notify when the forecast is likely to exceed it
  3. Lower Bedrock quotas in Service Quotas to cap usage
  4. Set an alarm on the CloudWatch token count metric
Show answer and explanation

Answer: B Create a budget in AWS Budgets and notify when the forecast is likely to exceed it

AWS Budgets can notify when actual or forecast amounts are likely to exceed the budget. [Other options] Weekly checks are manual and don't give early alerts. Token counts don't compare money to budget. Lower quotas block valid use too.

Q2. Since last month, Bedrock cost has suddenly tripled and the cause is unknown. What is the most appropriate next step?

  1. Check CloudWatch token counts per model and inference profile to find what grew
  2. Enable prompt caching in every app to cut input token cost
  3. Switch the most used model to a cheaper one and check next month's cost
  4. Check the Bedrock service total trend in Cost Explorer to find the month it rose
Show answer and explanation

Answer: A Check CloudWatch token counts per model and inference profile to find what grew

First identify the cause by measurement. Token counts per model, inference profile and period show which workload grew. [Other options] A service total doesn't show which workload grew. Switching models or adding caching before finding the cause can't confirm effect or quality impact.

Reference links

Lessons are written by this site based on public information and do not guarantee exam content. Product specifications change, so check the latest information at the reference links.

Notes (unofficial, original questions, no pass guarantee)