Data Engineering
Data engineering answers two questions — where the data lives and how it gets there. An analyst does not build pipelines, but must understand their design, otherwise they cannot explain why a metric lags a day or why a source is missing from a mart.
The topic is easy to fail on terminology. Schema-on-write versus schema-on-read, ETL versus ELT, log-based versus polling CDC — these pairs are easy to swap, and that is exactly where you get caught. Below is a map that untangles them.
Topic map
- Storage architectures — warehouse, lake, and lakehouse, schema-on-write versus schema-on-read, and medallion layers.
- ETL vs ELT — where transformation happens, why ELT beats ETL, and how CDC captures changes incrementally.
Common traps
| Mistake | Consequence |
|---|---|
| Swapping schema-on-write (warehouse) and schema-on-read (lake) | A wrong description of the whole storage architecture |
| Treating the lakehouse as just a smaller warehouse | Misses the point — ACID and table semantics over the lake |
| Calling ELT the legacy approach | The opposite — ELT is modern and leans on the warehouse's cheap MPP engine |
| Reversing the raw → cleansed → marts layer order | Raw is always first, marts last |
| Confusing the push and pull models of CDC | Log-based is push with low lag, not polling |
| Equating CDC with a full table reload | The whole point is lost — capturing only changed rows |
Interview relevance
Data engineering is asked to check whether an analyst understands the layer under their dashboards. A strong candidate untangles the term pairs and explains them through a trade-off, not a memorized definition — e.g. "ELT loads raw and transforms in the warehouse because warehouse compute got cheap".
Typical checks:
- The difference between warehouse, lake, and lakehouse and what schema-on-write/read means.
- Why data layers exist and in what order they run.
- ETL versus ELT — where transformation happens and when each fits.
- What CDC is and why log-based push loads the source less than polling.
Common wrong answer: "ETL and ELT are the same thing, the letters are just swapped". In fact the word order is the stage order: in ELT loading precedes transformation, and the transformation is done by the warehouse itself.