Skip to main content
Data Dare data transformation L4 sticker: the findi360 mascot analysing Enterprise Data Transformation and Performance Lab.

DD-TRF-004

Enterprise Data Transformation and Performance Lab

Decide what each monthly partition adds that the export does not yet hold, which partitions drifted, and whether the pipeline reconciles at scale.

  • Data Transformation
  • L4 · PARQUET
  • ~6,179,937 rows
  • 18 tables
  • 60 questions
Get the data 60 questions

Get the data

v1.0 · 6 files

Files are for members. A free account takes a minute, and you come straight back here.

Create a free account Sign in
  • Question Book PDF DD-TRF-004_Question_Book.pdf 1.3 MB · v1.0
  • Dataset Documentation / Data Dictionary XLSX DD-TRF-004_Data_Dictionary.xlsx 14.1 KB · v1.0
  • Business Introduction PDF DD-TRF-004_Business_Introduction.pdf 614.6 KB · v1.0
  • Relationship Diagram PDF DD-TRF-004_Relationship_Model.pdf 616.1 KB · v1.0
  • Raw fact data DD-TRF-004_FactDataPipelineEvents.parquet 139.3 MB · v1.0
  • Dimension / supporting data DD-TRF-004_Supporting_Tables.zip 16.3 MB · v1.0

Questions

Open one to discuss it
  1. Q001 Easy Rows in the export file
  2. Q002 Easy Exact duplicate rows
  3. Q003 Easy Conflicting duplicate IDs
  4. Q004 Easy Blank customer codes
  5. Q005 Easy Amounts stored as text
  6. Q006 Easy Dates not in ISO format
  7. Q007 Medium Clean amount by month
  8. Q008 Medium Valid rows by month
  9. Q009 Medium Distinct customers by month
  10. Q010 Medium Exact duplicates by month
  11. Q011 Medium Conflicting duplicates by month
  12. Q012 Medium Blank customer codes by month
  13. Q013 Medium Text amounts by month
  14. Q014 Medium Non-ISO dates by month
  15. Q015 Medium Stray-case codes by month
  16. Q016 Medium Unknown products by month
  17. Q017 Medium Return lines by month
  18. Q018 Medium Value of returns by month
  19. Q019 Medium Year-on-year change in clean amount
  20. Q020 Medium Clean amount by category
  21. Q021 Medium Clean amount by region
  22. Q022 Hard Regions driving the year-on-year change
  23. Q023 Hard Category standardisation bridge
  24. Q024 Hard Reconcile batch Partition_2025_01 to the export
  25. Q025 Hard Reconcile batch Partition_2025_02 to the export
  26. Q026 Hard Reconcile batch Partition_2025_03 to the export
  27. Q027 Hard Reconcile batch Partition_2025_04 to the export
  28. Q028 Hard Reconcile batch Partition_2025_05 to the export
  29. Q029 Hard Reconcile batch Partition_2025_06 to the export
  30. Q030 Hard Reconcile batch Partition_2025_07 to the export
  31. Q031 Hard Reconcile batch Partition_2025_08 to the export
  32. Q032 Hard Reconcile batch Partition_2025_09 to the export
  33. Q033 Hard Reconcile batch Partition_2025_10 to the export
  34. Q034 Hard Reconcile batch Partition_2025_11 to the export
  35. Q035 Hard Reconcile batch Partition_2025_12 to the export
  36. Q036 Hard Sales by customer tier as at transaction date
  37. Q037 Hard Rows breaking the exception rules
  38. Q038 Hard Monthly rates against a flat rate
  39. Q039 Hard Top ten products before and after cleaning
  40. Q040 Very Hard Checksum of the final table
  41. Q041 Very Hard Conflicting duplicates: amount at stake
  42. Q042 Very Hard Survivorship by latest load
  43. Q043 Very Hard Customers collapsed by alias mapping
  44. Q044 Very Hard Sales re-tiered by the SCD change
  45. Q045 Very Hard Schema drift in batch Partition_2025_03
  46. Q046 Very Hard Schema drift in batch Partition_2025_07
  47. Q047 Very Hard Schema drift in batch Partition_2025_11
  48. Q048 Very Hard Incremental load across all batches
  49. Q049 Very Hard Incremental rows from partition Partition_2025_01
  50. Q050 Very Hard Incremental rows from partition Partition_2025_02
  51. Q051 Very Hard Incremental rows from partition Partition_2025_03
  52. Q052 Very Hard Incremental rows from partition Partition_2025_04
  53. Q053 Very Hard Incremental rows from partition Partition_2025_05
  54. Q054 Very Hard Incremental rows from partition Partition_2025_06
  55. Q055 Very Hard Incremental rows from partition Partition_2025_07
  56. Q056 Very Hard Incremental rows from partition Partition_2025_08
  57. Q057 Very Hard Incremental rows from partition Partition_2025_09
  58. Q058 Very Hard Incremental rows from partition Partition_2025_10
  59. Q059 Very Hard Incremental rows from partition Partition_2025_11
  60. Q060 Very Hard Incremental rows from partition Partition_2025_12

General discussion

Anything about this dataset that is not one question — loading the files, which tool, how you set it up.

A free account takes a minute. Your draft is kept while you sign in.

Create a free account Sign in