Course 7 of 11

Big Data with Spark

RDDs & DataFrames, Transformations, Optimisation, and more.

Yiga AI

Intermediate en

Intermediate · Course · 4 weeks at 5 hrs/week

Duration: 4 weeks
Effort: 5 hrs/week
Lessons: 20
Hands-on exercises: 20
Projects: 1
Language: en

What you’ll learn

Skills you can put to work.

Data Engineering Foundations

Advanced SQL for Engineers

Python for Data Engineering

Data Warehousing

Syllabus

Every module, step by step.

4 modules · 25 lessons · 17 hours · 1 graded project

1000 minutes of guided lesson time.

Module 1 · 6 lessons · 3.7 hoursRDDs & DataFramesModule 1 of 4 in Big Data with Spark.

Overview and set-up
Overview and set-up — RDDs & DataFrames in Big Data with Spark.
Lesson
40 min
Core concepts
Core concepts — RDDs & DataFrames in Big Data with Spark.
Lesson · Locked
40 min
Worked example
Worked example — RDDs & DataFrames in Big Data with Spark.
Lesson · Locked
40 min
Hands-on practice
Hands-on practice — RDDs & DataFrames in Big Data with Spark.
Lesson · Locked
40 min
Common pitfalls
Common pitfalls — RDDs & DataFrames in Big Data with Spark.
Lesson · Locked
40 min
Module quiz: RDDs & DataFrames
Check what you learned in RDDs & DataFrames before moving on.
Quiz · Locked
20 min

Module 2 · 6 lessons · 3.7 hours LockedTransformationsModule 2 of 4 in Big Data with Spark.

Overview and set-up
Overview and set-up — Transformations in Big Data with Spark.
Lesson · Locked
40 min
Core concepts
Core concepts — Transformations in Big Data with Spark.
Lesson · Locked
40 min
Worked example
Worked example — Transformations in Big Data with Spark.
Lesson · Locked
40 min
Hands-on practice
Hands-on practice — Transformations in Big Data with Spark.
Lesson · Locked
40 min
Common pitfalls
Common pitfalls — Transformations in Big Data with Spark.
Lesson · Locked
40 min
Module quiz: Transformations
Check what you learned in Transformations before moving on.
Quiz · Locked
20 min

Module 3 · 6 lessons · 3.7 hours LockedOptimisationModule 3 of 4 in Big Data with Spark.

Overview and set-up
Overview and set-up — Optimisation in Big Data with Spark.
Lesson · Locked
40 min
Core concepts
Core concepts — Optimisation in Big Data with Spark.
Lesson · Locked
40 min
Worked example
Worked example — Optimisation in Big Data with Spark.
Lesson · Locked
40 min
Hands-on practice
Hands-on practice — Optimisation in Big Data with Spark.
Lesson · Locked
40 min
Common pitfalls
Common pitfalls — Optimisation in Big Data with Spark.
Lesson · Locked
40 min
Module quiz: Optimisation
Check what you learned in Optimisation before moving on.
Quiz · Locked
20 min

Module 4 · 7 lessons · 5.7 hours LockedCluster basicsModule 4 of 4 in Big Data with Spark.

Overview and set-up
Overview and set-up — Cluster basics in Big Data with Spark.
Lesson · Locked
40 min
Core concepts
Core concepts — Cluster basics in Big Data with Spark.
Lesson · Locked
40 min
Worked example
Worked example — Cluster basics in Big Data with Spark.
Lesson · Locked
40 min
Hands-on practice
Hands-on practice — Cluster basics in Big Data with Spark.
Lesson · Locked
40 min
Common pitfalls
Common pitfalls — Cluster basics in Big Data with Spark.
Lesson · Locked
40 min
Module quiz: Cluster basics
Check what you learned in Cluster basics before moving on.
Quiz · Locked
20 min
Graded project: Big Data with Spark
Apply everything from Big Data with Spark in one graded, portfolio-ready build.
Graded project · Locked
120 min

Applied learning project

Make the learning visible.

Use Graded project: Big Data with Spark to apply this course to a practical challenge. You’ll finish with a clear solution and the reasoning behind your choices.

Apply the course’s core skillMake and explain practical decisionsCreate evidence you can share

Your instructor

Grace A.

AI for professions

Shows working professionals how to use AI responsibly inside everyday workflows.

“The examples felt relevant to public service and worked on the connection I actually have.”
Samuel Mwangi
Public service officer, Nairobi

Continue with

Your next best step.

Following the order of Data Engineer.

See the full chain

Next lesson

Overview and set-up

Big Data with Spark

40 min · Resume

Course 8 in the track

dbt & Analytics Engineering

Models, Tests, Docs, and more.

Intermediate · Course · 4 weeks at 5 hrs/week

Course 9 in the track

Workflow Orchestration with Airflow

DAGs, Operators, Scheduling, and more.

Intermediate · Course · 4 weeks at 5 hrs/week

Course 10 in the track

Data Warehousing

Dimensional modelling, BigQuery/Snowflake, Loading, and more.

Intermediate · Course · 4 weeks at 5 hrs/week

Learner reviews

What learners say.

Write a review

Loading reviews…

Previous courseData Engineering Foundations Next coursedbt & Analytics Engineering

Keep going

Other courses in this track.

View full track

Data

16 lessons · 14h · Advanced

Streaming with Kafka

Topics, Producers/consumers, Stream processing, and more.

Data

20 lessons · 16.7h · Intermediate

Cloud Data Platforms

AWS/GCP/Azure services, Storage, IAM, and more.

Data

16 lessons · 14h · Intermediate

Python for Data Engineering

ETL scripts, Logging, Testing, and more.

Advanced SQL for Engineers cover illustration

Data

21 lessons · 16.8h · Intermediate

Advanced SQL for Engineers

Performance, Partitioning, Transactions, and more.