Skip to content

NaN filing between partitions with overlap

Muhammad Zain Sohail requested to merge flash-lazy2 into main

Created by: zain-sohail

Created a new branch because the one in PR #162 has too many other comments. I think after adding a test, this is good to go.

Changes from the other PR:

  • removed progress bar as it has a larger overhead than just calling len
  • removed the compute command because len directly gives the metadata access to the num_rows.

To be done:

adding tests with multiple files

  • partial data
  • one fully nan file (cols)
  • consective empty files

last one might fail so we would need null metadata for a more comprehensive treatment https://github.com/OpenCOMPES/sed/pull/162#discussion_r1350242308

also might make sense to keep this as a seperate function or even a generic method that other loaders can use as I think it's pretty useful

Merge request reports

Loading