Streaming with Kafka
Topics, Producers/consumers, Stream processing, and more.
Course 6 of 11
Roles, Architectures, Batch vs streaming, and more.
Yiga AI
Beginner · Course · 3 weeks at 5 hrs/week
What you’ll learn
Data Engineering Foundations
Advanced SQL for Engineers
Python for Data Engineering
Data Warehousing
Syllabus
4 modules · 21 lessons · 14 hours · 1 graded project
840 minutes of guided lesson time.
Sign in and enrol to unlock every lesson. The first lesson is open as a preview.
Overview and set-up
Overview and set-up — Roles in Data Engineering Foundations.
Lesson
Core concepts
Core concepts — Roles in Data Engineering Foundations.
Lesson · Locked
Worked example
Worked example — Roles in Data Engineering Foundations.
Lesson · Locked
Hands-on practice
Hands-on practice — Roles in Data Engineering Foundations.
Lesson · Locked
Module quiz: Roles
Check what you learned in Roles before moving on.
Quiz · Locked
Overview and set-up
Overview and set-up — Architectures in Data Engineering Foundations.
Lesson · Locked
Core concepts
Core concepts — Architectures in Data Engineering Foundations.
Lesson · Locked
Worked example
Worked example — Architectures in Data Engineering Foundations.
Lesson · Locked
Hands-on practice
Hands-on practice — Architectures in Data Engineering Foundations.
Lesson · Locked
Module quiz: Architectures
Check what you learned in Architectures before moving on.
Quiz · Locked
Overview and set-up
Overview and set-up — Batch vs streaming in Data Engineering Foundations.
Lesson · Locked
Core concepts
Core concepts — Batch vs streaming in Data Engineering Foundations.
Lesson · Locked
Worked example
Worked example — Batch vs streaming in Data Engineering Foundations.
Lesson · Locked
Hands-on practice
Hands-on practice — Batch vs streaming in Data Engineering Foundations.
Lesson · Locked
Module quiz: Batch vs streaming
Check what you learned in Batch vs streaming before moving on.
Quiz · Locked
Overview and set-up
Overview and set-up — Tooling map in Data Engineering Foundations.
Lesson · Locked
Core concepts
Core concepts — Tooling map in Data Engineering Foundations.
Lesson · Locked
Worked example
Worked example — Tooling map in Data Engineering Foundations.
Lesson · Locked
Hands-on practice
Hands-on practice — Tooling map in Data Engineering Foundations.
Lesson · Locked
Module quiz: Tooling map
Check what you learned in Tooling map before moving on.
Quiz · Locked
Graded project: Data Engineering Foundations
Apply everything from Data Engineering Foundations in one graded, portfolio-ready build.
Graded project · Locked
Applied learning project
Use Graded project: Data Engineering Foundations to apply this course to a practical challenge. You’ll finish with a clear solution and the reasoning behind your choices.

Your instructor
Software & cloud
Helps new developers build reliable products through clear explanations and guided practice.

“I use what I learned in my work now, not in some distant future project.”
Continue with
Following the order of Data Engineer.
Next lesson
Overview and set-up
Data Engineering Foundations
40 min · Resume
Course 7 in the track
Big Data with Spark
RDDs & DataFrames, Transformations, Optimisation, and more.
Intermediate · Course · 4 weeks at 5 hrs/week
Course 8 in the track
dbt & Analytics Engineering
Models, Tests, Docs, and more.
Intermediate · Course · 4 weeks at 5 hrs/week
Course 9 in the track
Workflow Orchestration with Airflow
DAGs, Operators, Scheduling, and more.
Intermediate · Course · 4 weeks at 5 hrs/week
Learner reviews
Sign in to share your experience with this course.
Sign in to reviewLoading reviews…
Keep going
Topics, Producers/consumers, Stream processing, and more.
AWS/GCP/Azure services, Storage, IAM, and more.
ETL scripts, Logging, Testing, and more.
Performance, Partitioning, Transactions, and more.