Messy dataset to practise data cleaning in Power Query, Python or SQL
DATA DARE's Data Transformation sets on findi360 are free, tool-neutral messy business-operations datasets built for cleaning and reshaping practice — in Power Query, Python (pandas or Polars) or SQL — with expected results you can check against: row counts, error counts, unmatched keys, reconciled totals and table shapes. Four levels, from a single untidy file to an enterprise package of millions of rows.
The four levels
- L1 — basic cleaning and shaping: types, blanks, duplicates, one file.
- L2 — multi-file transformation and data quality: append, merge, validate across files.
- L3 — complex joins, exceptions and transformation logic: unmatched keys, many-to-many, rules with exceptions.
- L4 — enterprise data transformation and performance: partitioned Parquet, millions of rows, schema drift.
What the questions ask
Clean, reshape, merge, append, validate, unpivot, de-duplicate, map and handle a schema that changes between files. Each answer is a number or a table — how many rows survive, how many keys fail to match, what the reconciled total is — so a Power Query solution and a pandas solution are marked the same way.
Which tool
Power Query (M) is the natural first tool for L1 and L2, because that is where finance people meet transformation. From L3 the same questions reward Python or SQL, and at L4 they almost require them. The free recording below works through the L4 package in Python.
The sets live now
- Basic Data Cleaning and Shaping DD-TRF-001 L1 — XLSX / ~10,000 rows Data Transformation Clean one dirty transaction export: fix types, remove duplicates, standardise codes, map regions and unpivot a wide target sheet.
- Multi-File Transformation and Data Quality DD-TRF-002 L2 — CSV / ~30,000 rows Data Transformation Combine four branch files with schema differences, reconcile them to the central export, validate keys and unpivot monthly targets.
- Complex Joins, Exceptions and Transformation Logic DD-TRF-003 L3 — Parquet / ~500,000 rows Data Transformation Join effective-dated customer attributes, map aliases, convert currencies, apply exception rules and reconcile six monthly batches to the export.
- Enterprise Data Transformation and Performance Lab DD-TRF-004 L4 — Parquet / ~3,000,000 rows Data Transformation Run the same transformation logic over a multi-million-row export and twelve monthly partitions with schema drift, incremental loads and survivorship rules — in...
The course
- Data Transformation in Python — At Scale Free recording One export of 6.18 million dirty rows, cleaned in Python and reconciled to a total you can defend. For Power Query users who have never written Python. Amounts stored as text, thre...