Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ reproducing Microsoft Learn pages.

## Project status

The project is in its first foundation milestone. The repository currently
provides separate public reader and private administration application
factories, environment-validated settings, automated boundary tests, and CI.
Reader features and DP-700 content will be added through the public roadmap.
The project is in its first reader milestone. The repository provides separate
public reader and private administration application factories, a validated
exam-agnostic book contract, the current DP-700 outline, sanitized Markdown
rendering, a responsive reader shell, automated boundary tests, and CI. Reader
preferences, local progress, annotations, and offline support remain roadmap
work.

## Architecture boundary

Expand Down Expand Up @@ -62,6 +64,10 @@ project is not affiliated with or endorsed by Microsoft. Microsoft owns its
learning material; this project links to authoritative sources and publishes
original study-oriented writing under the repository license.

The DP-700 manifest records the official skills blueprint effective July 21,
2026. See [`docs/content-model.md`](docs/content-model.md) for stable identifier,
ordering, and source-mapping rules.

## Contributing and security

Work is tracked through GitHub issues and reviewed pull requests. Read
Expand Down
137 changes: 137 additions & 0 deletions content/published/dp700/book.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
id: dp700
exam_code: DP-700
title: Implementing Data Engineering Solutions Using Microsoft Fabric
blueprint_effective_date: 2026-07-21
blueprint_url: https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700
sources:
- id: dp700-study-guide
title: Official DP-700 study guide
url: https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700
domains:
- id: implement-manage
title: Implement and manage an analytics solution
weight: "30–35%"
chapters:
- id: workspace-settings
slug: workspace-settings
title: Configure Microsoft Fabric workspace settings
content_path: workspace-settings.md
status: draft
source_ids: [dp700-study-guide]
objectives:
- {id: implement.workspace.spark, title: Configure Spark workspace settings}
- {id: implement.workspace.domain, title: Configure domain workspace settings}
- {id: implement.workspace.onelake, title: Configure OneLake workspace settings}
- {id: implement.workspace.airflow, title: Configure Apache Airflow workspace settings}
- id: lifecycle-management
slug: lifecycle-management
title: Implement lifecycle management in Fabric
content_path: lifecycle-management.md
source_ids: [dp700-study-guide]
objectives:
- {id: implement.lifecycle.version-control, title: Configure version control}
- {id: implement.lifecycle.database-projects, title: Implement database projects}
- {id: implement.lifecycle.deployment-pipelines, title: Create and configure deployment pipelines}
- id: security-governance
slug: security-governance
title: Configure security and governance
content_path: security-governance.md
source_ids: [dp700-study-guide]
objectives:
- {id: implement.security.workspace-access, title: Implement workspace-level access controls}
- {id: implement.security.item-access, title: Implement item-level access controls}
- {id: implement.security.data-access, title: "Implement row-level, column-level, object-level, and folder/file-level access controls"}
- {id: implement.security.masking, title: Implement dynamic data masking}
- {id: implement.security.sensitivity, title: Apply sensitivity labels to items}
- {id: implement.security.endorsement, title: Endorse items}
- {id: implement.security.audit, title: Implement and use Microsoft Fabric audit logs}
- {id: implement.security.onelake, title: Configure and implement OneLake security}
- id: orchestration
slug: orchestration
title: Orchestrate processes
content_path: orchestration.md
source_ids: [dp700-study-guide]
objectives:
- {id: implement.orchestration.choose-tool, title: "Choose between Dataflow Gen2, a pipeline, and a notebook"}
- {id: implement.orchestration.triggers, title: Design and implement schedules and event-based triggers}
- {id: implement.orchestration.patterns, title: "Implement orchestration patterns with notebooks and pipelines, including parameters and dynamic expressions"}
- id: ingest-transform
title: Ingest and transform data
weight: "30–35%"
chapters:
- id: loading-patterns
slug: loading-patterns
title: Design and implement loading patterns
content_path: loading-patterns.md
source_ids: [dp700-study-guide]
objectives:
- {id: ingest.loading.full-incremental, title: Design and implement full and incremental data loads}
- {id: ingest.loading.dimensional, title: Prepare data for loading into a dimensional model}
- {id: ingest.loading.streaming, title: Design and implement a loading pattern for streaming data}
- id: batch-data
slug: batch-data
title: Ingest and transform batch data
content_path: batch-data.md
source_ids: [dp700-study-guide]
objectives:
- {id: ingest.batch.store, title: Choose an appropriate data store}
- {id: ingest.batch.transform-tool, title: "Choose between Dataflows Gen2, notebooks, KQL, and T-SQL for data transformation"}
- {id: ingest.batch.shortcuts, title: Create and manage OneLake shortcuts}
- {id: ingest.batch.mirroring, title: Implement mirroring}
- {id: ingest.batch.pipelines, title: Ingest data by using pipelines}
- {id: ingest.batch.languages, title: "Transform data by using PySpark, SQL, and KQL"}
- {id: ingest.batch.denormalize, title: Denormalize data}
- {id: ingest.batch.aggregate, title: Group and aggregate data}
- {id: ingest.batch.data-quality, title: "Handle duplicate, missing, and late-arriving data"}
- id: streaming-data
slug: streaming-data
title: Ingest and transform streaming data
content_path: streaming-data.md
source_ids: [dp700-study-guide]
objectives:
- {id: ingest.streaming.engine, title: Choose an appropriate streaming engine}
- {id: ingest.streaming.native-shortcut, title: Choose between native tables and OneLake shortcuts in Real-Time Intelligence}
- {id: ingest.streaming.acceleration, title: Choose between Query acceleration for OneLake shortcuts and standard OneLake shortcuts in Real-Time Intelligence}
- {id: ingest.streaming.eventstreams, title: Process data by using Eventstreams}
- {id: ingest.streaming.spark, title: Process data by using Spark structured streaming}
- {id: ingest.streaming.kql, title: Process data by using KQL}
- {id: ingest.streaming.windows, title: Create windowing functions}
- id: monitor-optimize
title: Monitor and optimize an analytics solution
weight: "30–35%"
chapters:
- id: monitor-items
slug: monitor-items
title: Monitor Fabric items
content_path: monitor-items.md
source_ids: [dp700-study-guide]
objectives:
- {id: monitor.items.ingestion, title: Monitor data ingestion}
- {id: monitor.items.transformation, title: Monitor data transformation}
- {id: monitor.items.semantic-refresh, title: Monitor semantic model refresh}
- {id: monitor.items.alerts, title: Configure alerts}
- id: resolve-errors
slug: resolve-errors
title: Identify and resolve errors
content_path: resolve-errors.md
source_ids: [dp700-study-guide]
objectives:
- {id: monitor.errors.pipeline, title: Identify and resolve pipeline errors}
- {id: monitor.errors.dataflow, title: Identify and resolve Dataflow Gen2 errors}
- {id: monitor.errors.notebook, title: Identify and resolve notebook errors}
- {id: monitor.errors.eventhouse, title: Identify and resolve Eventhouse errors}
- {id: monitor.errors.eventstream, title: Identify and resolve Eventstream errors}
- {id: monitor.errors.tsql, title: Identify and resolve T-SQL errors}
- {id: monitor.errors.shortcut, title: Identify and resolve OneLake shortcut errors}
- id: optimize-performance
slug: optimize-performance
title: Optimize performance
content_path: optimize-performance.md
source_ids: [dp700-study-guide]
objectives:
- {id: monitor.optimize.lakehouse, title: Optimize a Lakehouse table}
- {id: monitor.optimize.pipeline, title: Optimize a pipeline}
- {id: monitor.optimize.warehouse, title: Optimize a data warehouse}
- {id: monitor.optimize.realtime, title: Optimize Eventstreams and Eventhouses}
- {id: monitor.optimize.spark, title: Optimize Spark performance}
- {id: monitor.optimize.query, title: Optimize query performance}
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/batch-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Ingest and transform batch data

<!-- block-id: placeholder-orientation -->
This chapter will connect store selection, ingestion mechanisms, transformation languages, and data-quality handling into an end-to-end batch design.

## Planned study work

- Choose among shortcuts, mirroring, and pipeline copies.
- Compare Dataflows Gen2, notebooks, KQL, and T-SQL.
- Handle duplicate, missing, and late-arriving records explicitly.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/lifecycle-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Implement lifecycle management in Fabric

<!-- block-id: placeholder-orientation -->
This chapter will connect version control, database projects, and deployment pipelines into one reviewable DEV-to-production lifecycle.

## Planned study work

- Compare Git integration with deployment pipelines.
- Trace how database project changes are built and reviewed.
- Practice selecting the correct promotion mechanism for a scenario.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/loading-patterns.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Design and implement loading patterns

<!-- block-id: placeholder-orientation -->
This chapter will model full, incremental, dimensional, and streaming loads as choices driven by source behavior, latency, correctness, and recovery needs.

## Planned study work

- Compare watermarks, change tracking, and full reloads.
- Prepare facts and dimensions for reliable loading.
- Design a recoverable streaming ingestion path.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/monitor-items.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Monitor Fabric items

<!-- block-id: placeholder-orientation -->
This chapter will develop an evidence path across ingestion, transformation, semantic model refresh, and actionable alerts.

## Planned study work

- Identify the correct monitoring surface for each item.
- Correlate upstream ingestion with downstream refresh behavior.
- Configure alerts around operationally meaningful conditions.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/optimize-performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Optimize performance

<!-- block-id: placeholder-orientation -->
This chapter will separate storage layout, orchestration, compute, and query optimization so that each performance change is tied to measured evidence.

## Planned study work

- Diagnose before selecting an optimization.
- Compare Lakehouse, warehouse, Spark, and real-time tuning levers.
- Validate improvements against a repeatable workload.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/orchestration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Orchestrate processes

<!-- block-id: placeholder-orientation -->
This chapter will compare Dataflow Gen2, pipelines, and notebooks, then develop schedules, event triggers, parameters, and dynamic orchestration patterns.

## Planned study work

- Select an orchestration tool from workload constraints.
- Trace parameters through a multi-step pipeline.
- Compare scheduled and event-driven execution.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/resolve-errors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Identify and resolve errors

<!-- block-id: placeholder-orientation -->
This chapter will use a consistent diagnose-isolate-correct-verify loop across pipelines, Dataflows Gen2, notebooks, Eventhouse, Eventstreams, T-SQL, and shortcuts.

## Planned study work

- Map common failures to their best diagnostic evidence.
- Separate configuration, identity, data, and runtime causes.
- Turn recurring failures into regression checks.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/security-governance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Configure security and governance

<!-- block-id: placeholder-orientation -->
This chapter will distinguish workspace, item, data, and OneLake security boundaries, then connect them to labels, endorsements, masking, and audit evidence.

## Planned study work

- Build a role-and-scope decision table.
- Compare row, column, object, and file-level controls.
- Work through governance scenarios without conflating discovery and authorization.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
14 changes: 14 additions & 0 deletions content/published/dp700/chapters/streaming-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Ingest and transform streaming data

<!-- block-id: placeholder-orientation -->
This chapter will compare streaming engines, Real-Time Intelligence storage choices, Eventstreams, Spark structured streaming, KQL, and window semantics.

## Planned study work

- Select an engine from latency and processing requirements.
- Compare native tables with standard and accelerated shortcuts.
- Reason about tumbling, hopping, and sliding windows.

## Source

See the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700).
42 changes: 42 additions & 0 deletions content/published/dp700/chapters/workspace-settings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Configure Microsoft Fabric workspace settings

<!-- block-id: orientation -->
Workspace settings are control points. They shape how a team uses compute, organizes ownership, reaches shared data, and schedules orchestration. For the exam, focus on **who controls a setting, what scope it affects, and what operational consequence follows**.

## Start with the boundary

<!-- block-id: boundary-model -->
A workspace is a collaboration and security boundary, but not every behavior is controlled at workspace scope. Some defaults inherit from capacity or tenant administration. Before changing a setting, identify the governing scope and whether the workspace administrator can override it.

| Setting family | Primary concern | Exam question to ask |
| --- | --- | --- |
| Spark | Compute defaults and runtime behavior | Does this change the workspace default or only one session? |
| Domain | Business ownership and discovery | Is the workspace assigned to the correct data domain? |
| OneLake | How data is exposed and accessed | Does the change affect access, discoverability, or data movement? |
| Apache Airflow | Managed orchestration configuration | Who owns connections, schedules, and operational monitoring? |

## A practical decision sequence

<!-- block-id: decision-sequence -->
1. Identify the required outcome: performance, governance, access, or orchestration.
2. Locate the narrowest administrative scope that owns the behavior.
3. Check inheritance and override rules before changing a default.
4. Validate the effect with a representative workload rather than assuming the setting is isolated.
5. Record the operational owner and a rollback path.

## Exam distinction

<!-- block-id: exam-distinction -->
Do not treat all workspace settings as interchangeable toggles. Spark settings primarily influence compute behavior; domain assignment communicates business organization; OneLake settings influence the shared data plane; and Apache Airflow settings support managed workflow orchestration. A scenario usually gives clues about **scope**, **owner**, and **effect**.

## Active recall

<!-- block-id: active-recall -->
- Which settings would you examine first when every notebook in a workspace starts with an unsuitable Spark default?
- Why is assigning a workspace to a domain different from granting access to the workspace?
- What evidence would you collect before changing a setting that may affect multiple workloads?

## Source

<!-- block-id: official-source -->
Use the [official DP-700 study guide](https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700) as the authoritative skills outline. This chapter is original study-oriented writing and will be expanded with setting-specific Microsoft Learn sources in the content-authoring milestone.
46 changes: 46 additions & 0 deletions docs/content-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Book content model

Study Reader separates authored chapters from application code. A published
book is a directory containing `book.yaml` and a `chapters/` directory of
Markdown files.

## Stable identifiers

- Book, domain, chapter, source, and objective identifiers MUST be lowercase,
human-readable, and stable after publication.
- Chapter slugs MUST be unique within a book and MUST remain stable when a title
changes.
- Objective identifiers SHOULD describe their hierarchy, such as
`ingest.streaming.windows`.
- An explicit `<!-- block-id: durable-name -->` marker assigns a durable anchor
to the next top-level Markdown block. Authors SHOULD use explicit block IDs
for important passages that may receive bookmarks, highlights, or notes.
- Blocks without explicit markers receive deterministic section-and-position
identifiers. Those fallbacks survive ordinary wording edits but MAY change if
blocks are inserted or reordered in the same section.

## Ordering

YAML list order is authoritative. Domain order follows the official skills
outline. Chapter order follows the subdomains within each domain. Objective
order follows the bullet order in the effective official blueprint. The reader
MUST NOT infer order from identifiers or filenames.

## Sources and ownership

Every chapter MUST map to at least one source declared in its book manifest.
The manifest records the official blueprint URL and effective date. Additional
chapter-specific sources will record retrieval dates and hashes in the refresh
milestone.

Microsoft Learn remains authoritative for Microsoft exam objectives and product
behavior. Markdown chapters MUST contain original study-oriented writing and
MUST NOT reproduce Microsoft Learn pages wholesale.

## Validation

Pydantic rejects duplicate identifiers, duplicate slugs, broken source
references, invalid paths, and unknown manifest fields. Content tests confirm
that every planned chapter file exists and maps at least one objective and one
source. Rendering tests verify sanitized HTML, stable anchors, external-link
behavior, and a reviewed golden fixture.
Loading
Loading