100% Latest & Updated Databricks Certified Data Engineer Associate Practice Test Questions, Exam Dumps & Verified Answers!
30 Days Free Updates, Instant Download!
Question 260
A heavily shuffled ETL join creates thousands of tiny shuffle partitions while tasks spend time scheduling. The team observes: The job has many near-empty shuffle tasks and disproportionate scheduler overhead. What is the most likely mechanism behind the observed defect?
- A standard view was used where materialization would meet the latency and freshness contract.
- The job counted raw messages without key-level deduplication.
- The default shuffle partition count was unsuitable for this workload.
- The transform interpreted local clock text without its source offset.
Explanation
Correct Answer
C:The default shuffle partition count was unsuitable for this workload. Compare runs while adjusting spark.sql.shuffle.partitions within tested bounds.
Incorrect Answers
A:If a busy BI dashboard repeatedly runs an expensive grouped query and tolerates refresh-based staleness, “A standard view was used where materialization would meet the latency and freshness contract” may be the appropriate response. The default shuffle partition count was unsuitable for this workload.
B:A click event feed contains retries with identical event_id and should count each business event once. In those circumstances, “The job counted raw messages without key-level deduplication” fits. The default shuffle partition count was unsuitable for this workload.
D:If a payroll bronze table has event timestamps saved as arbitrary text from two time zones, “The transform interpreted local clock text without its source offset” may be the appropriate response. The default shuffle partition count was unsuitable for this workload.
Question 261
A large join spills to disk repeatedly although executor memory pressure is high. Which approach most directly satisfies the requirement?
- Size and optimize the join and memory allocation after inspecting Spark UI evidence.
- Use a governed SQL view when live query-time recomputation is desired.
- Apply a business-rule assertion before promoting data to silver or gold.
- Use a left join starting from the account relation.
Explanation
Correct Answer
A:Size and optimize the join and memory allocation after inspecting Spark UI evidence. Spill bytes and wall time improve in a comparable run without OOM errors.
Incorrect Answers
B:When a finance dashboard reads a curated metric that should always reflect the latest base table without storing a separately maintained aggregate, “Use a governed SQL view when live query-time recomputation is desired” could be relevant. Size and optimize the join and memory allocation after inspecting Spark UI evidence.
C:If a payments silver table must never have negative approved transaction amounts, “Apply a business-rule assertion before promoting data to silver or gold” may be the appropriate response. Size and optimize the join and memory allocation after inspecting Spark UI evidence.
D:A subscriptions dataset must preserve every account even when no payment has arrived. In those circumstances, “Use a left join starting from the account relation” fits. Size and optimize the join and memory allocation after inspecting Spark UI evidence.
Question 262
A large join spills to disk repeatedly although executor memory pressure is high. Which configuration or engineering action would most directly implement the requirement?
- Partition by customer_id, order by timestamp and event sequence descending, and keep row_number=1.
- Review spill metrics and join strategy, then test supported executor memory or partition adjustments.
- Supply both key equality predicates to the Spark join condition.
- Implement an amount >= 0 validation and define an explicit fail or quarantine action.
Explanation
Correct Answer
B:Review spill metrics and join strategy, then test supported executor memory or partition adjustments. Spill bytes and wall time improve in a comparable run without OOM errors.
Incorrect Answers
A:When a customer dataset must retain the latest update per business key even when events arrive out of order, “Partition by customer_id, order by timestamp and event sequence descending, and keep row_number=1” could be relevant. Review spill metrics and join strategy, then test supported executor memory or partition adjustments.
C:A customer match requires both organization identifier and customer identifier, since a customer number is reused by different businesses. In those circumstances, “Supply both key equality predicates to the Spark join condition” fits. Review spill metrics and join strategy, then test supported executor memory or partition adjustments.
D:When a payments silver table must never have negative approved transaction amounts, “Implement an amount >= 0 validation and define an explicit fail or quarantine action” could be relevant. Review spill metrics and join strategy, then test supported executor memory or partition adjustments.
Question 263
A large join spills to disk repeatedly although executor memory pressure is high. Which test would establish that the change worked as intended?
- Orders with valid matching products appear once according to dimension key uniqueness; unmatched orders are absent.
- A known input yields the expected two columns and malformed records are accounted for.
- Spill bytes and wall time improve in a comparable run without OOM errors.
- Repeated dashboard queries read the materialized result and updates follow the refresh SLA.
Explanation
Correct Answer
C:Spill bytes and wall time improve in a comparable run without OOM errors.
Incorrect Answers
A:If a product-order dataset needs one enriched row only when a matching dimension key exists, “Orders with valid matching products appear once according to dimension key uniqueness; unmatched orders are absent” may be the appropriate response. Spill bytes and wall time improve in a comparable run without OOM errors.
B:A source field packs a city and postal code with a single pipe delimiter. In those circumstances, “A known input yields the expected two columns and malformed records are accounted for” fits. Spill bytes and wall time improve in a comparable run without OOM errors.
D:If a busy BI dashboard repeatedly runs an expensive grouped query and tolerates refresh-based staleness, “Repeated dashboard queries read the materialized result and updates follow the refresh SLA” may be the appropriate response. Spill bytes and wall time improve in a comparable run without OOM errors.
Question 264
A large join spills to disk repeatedly although executor memory pressure is high. The team observes: The same stage writes large temporary spill files every run. Which configuration or design gap best explains the symptom?
- The partition or memory design exceeds available per-task memory.
- The transformation used SELECT * or passed through all source fields.
- Deduplication kept arbitrary arrival order rather than event-version order.
- The transform stored the whole array as a scalar rather than expanding it.
Explanation
Correct Answer
A:The partition or memory design exceeds available per-task memory. Review spill metrics and join strategy, then test supported executor memory or partition adjustments.
Incorrect Answers
B:When an account table has three irrelevant vendor-internal columns that must not reach the published silver table, “The transformation used SELECT * or passed through all source fields” could be relevant. The partition or memory design exceeds available per-task memory.
C:If a customer dataset must retain the latest update per business key even when events arrive out of order, “Deduplication kept arbitrary arrival order rather than event-version order” may be the appropriate response. The partition or memory design exceeds available per-task memory.
D:A line-item DataFrame contains an array of product codes per order and downstream rows must be per product. In those circumstances, “The transform stored the whole array as a scalar rather than expanding it” fits. The partition or memory design exceeds available per-task memory.
Question 265
A Spark job has many tiny input partitions and inefficient parallel task utilization. Which architecture or engineering decision best fits this scenario?
- Evaluate the supported input and execution partitioning strategy for available parallelism.
- Apply a business-rule assertion before promoting data to silver or gold.
- Project only the explicitly approved fields rather than carrying the full source schema.
- Choose an inner join between order facts and the matching product dimension.
Explanation
Correct Answer
A:Evaluate the supported input and execution partitioning strategy for available parallelism. Comparable runs use useful parallel tasks with lower total overhead.
Incorrect Answers
B:If a payments silver table must never have negative approved transaction amounts, “Apply a business-rule assertion before promoting data to silver or gold” may be the appropriate response. Evaluate the supported input and execution partitioning strategy for available parallelism.
C:An account table has three irrelevant vendor-internal columns that must not reach the published silver table. In those circumstances, “Project only the explicitly approved fields rather than carrying the full source schema” fits. Evaluate the supported input and execution partitioning strategy for available parallelism.
D:When a product-order dataset needs one enriched row only when a matching dimension key exists, “Choose an inner join between order facts and the matching product dimension” could be relevant. Evaluate the supported input and execution partitioning strategy for available parallelism.
Question 266
A Spark job has many tiny input partitions and inefficient parallel task utilization. Which implementation step is most appropriate before production use?
- Implement an amount >= 0 validation and define an explicit fail or quarantine action.
- Group the relevant rows and apply approx_count_distinct to device_id.
- Inspect the current partitions and adjust repartition or coalesce based on stage metrics.
- Join accounts LEFT OUTER JOIN payments on account_id and preserve missing payment fields.
Explanation
Correct Answer
C:Inspect the current partitions and adjust repartition or coalesce based on stage metrics. Comparable runs use useful parallel tasks with lower total overhead.
Incorrect Answers
A:If a payments silver table must never have negative approved transaction amounts, “Implement an amount >= 0 validation and define an explicit fail or quarantine action” may be the appropriate response. Inspect the current partitions and adjust repartition or coalesce based on stage metrics.
B:A reporting table must count unique devices across billions of events with an acceptable approximation. In those circumstances, “Group the relevant rows and apply approx_count_distinct to device_id” fits. Inspect the current partitions and adjust repartition or coalesce based on stage metrics.
D:If a subscriptions dataset must preserve every account even when no payment has arrived, “Join accounts LEFT OUTER JOIN payments on account_id and preserve missing payment fields” may be the appropriate response. Inspect the current partitions and adjust repartition or coalesce based on stage metrics.
Question 267
A Spark job has many tiny input partitions and inefficient parallel task utilization. Which acceptance test provides the strongest evidence that the intended outcome has been achieved?
- The executed plan shows a broadcast join and no large dimension repartition overhead.
- The impossible reading is segregated, valid floats remain queryable, and counts reconcile with bronze.
- Comparable runs use useful parallel tasks with lower total overhead.
- The approximate count is within the accepted tolerance of an exact sample benchmark.
Explanation
Correct Answer
C:Comparable runs use useful parallel tasks with lower total overhead.
Incorrect Answers
A:A small static country table enriches a huge events DataFrame and a large shuffle must be avoided when eligible. In those circumstances, “The executed plan shows a broadcast join and no large dimension repartition overhead” fits. Comparable runs use useful parallel tasks with lower total overhead.
B:When an IoT bronze dataset contains temperatures as strings and an impossible reading that must not contaminate hourly gold summaries, “The impossible reading is segregated, valid floats remain queryable, and counts reconcile with bronze” could be relevant. Comparable runs use useful parallel tasks with lower total overhead.
D:A reporting table must count unique devices across billions of events with an acceptable approximation. In those circumstances, “The approximate count is within the accepted tolerance of an exact sample benchmark” fits. Comparable runs use useful parallel tasks with lower total overhead.
Question 268
A Spark job has many tiny input partitions and inefficient parallel task utilization. The team observes: Most available executors remain idle while a few oversized tasks run. Which root cause should be addressed first given this evidence?
- A materialized object was not refreshed and should have been a view for this need.
- Deduplication kept arbitrary arrival order rather than event-version order.
- Partitioning is too coarse for the available worker parallelism.
- The operation used left outer or omitted the anti-match predicate.
Explanation
Correct Answer
C:Partitioning is too coarse for the available worker parallelism. Inspect the current partitions and adjust repartition or coalesce based on stage metrics.
Incorrect Answers
A:When a finance dashboard reads a curated metric that should always reflect the latest base table without storing a separately maintained aggregate, “A materialized object was not refreshed and should have been a view for this need” could be relevant. Partitioning is too coarse for the available worker parallelism.
B:If a customer dataset must retain the latest update per business key even when events arrive out of order, “Deduplication kept arbitrary arrival order rather than event-version order” may be the appropriate response. Partitioning is too coarse for the available worker parallelism.
D:When a GDPR deletion feed requires excluding rows whose identifiers appear on a revoked-consent list, “The operation used left outer or omitted the anti-match predicate” could be relevant. Partitioning is too coarse for the available worker parallelism.
Question 269
A large fact-to-small-dimension join repeatedly performs an avoidable shuffle. Which technical design is most appropriate for the stated constraint?
- Apply a deliberate cross join to form a bounded Cartesian product.
- Validate currencies and join the approved conversion reference before writing standardized silver amounts.
- Consider an eligible broadcast join and the autoBroadcastJoinThreshold setting.
- Use a left join starting from the account relation.
Explanation
Correct Answer
C:Consider an eligible broadcast join and the autoBroadcastJoinThreshold setting. The execution plan reflects a broadcast strategy and the run improves without OOM.
Incorrect Answers
A:If a risk team needs all combinations of five scenario assumptions and six stress categories, “Apply a deliberate cross join to form a bounded Cartesian product” may be the appropriate response. Consider an eligible broadcast join and the autoBroadcastJoinThreshold setting.
B:A marketing bronze feed arrives with mixed currency columns, but conversion rates are supplied in a governed reference table. In those circumstances, “Validate currencies and join the approved conversion reference before writing standardized silver amounts” fits. Consider an eligible broadcast join and the autoBroadcastJoinThreshold setting.
D:If a subscriptions dataset must preserve every account even when no payment has arrived, “Use a left join starting from the account relation” may be the appropriate response. Consider an eligible broadcast join and the autoBroadcastJoinThreshold setting.
Question 270
A large fact-to-small-dimension join repeatedly performs an avoidable shuffle. Which action should the engineer perform to establish the intended control?
- Measure dimension size and tune the broadcast threshold or apply an eligible hint only if memory fits.
- Use select on an allowlist or drop the disallowed columns before writing the table.
- Use split with the literal delimiter, trim both elements, and handle malformed input explicitly.
- Create the appropriate Unity Catalog view with an explicit stable SELECT.
Explanation
Correct Answer
A:Measure dimension size and tune the broadcast threshold or apply an eligible hint only if memory fits. The execution plan reflects a broadcast strategy and the run improves without OOM.
Incorrect Answers
B:When an account table has three irrelevant vendor-internal columns that must not reach the published silver table, “Use select on an allowlist or drop the disallowed columns before writing the table” could be relevant. Measure dimension size and tune the broadcast threshold or apply an eligible hint only if memory fits.
C:If a source field packs a city and postal code with a single pipe delimiter, “Use split with the literal delimiter, trim both elements, and handle malformed input explicitly” may be the appropriate response. Measure dimension size and tune the broadcast threshold or apply an eligible hint only if memory fits.
D:A finance dashboard reads a curated metric that should always reflect the latest base table without storing a separately maintained aggregate. In those circumstances, “Create the appropriate Unity Catalog view with an explicit stable SELECT” fits. Measure dimension size and tune the broadcast threshold or apply an eligible hint only if memory fits.
Certified Data Engineer Associate Practice Tests