[nabobalis]

PR Description

I am recreating a few cubes to replace the data and I see that it spikes in RAM use over 20GB for some of my cubes. This is an attempt to reduce that. Having spoken to Alasdair, this might be a terrible solution for smaller cubes.

TODO:

• Rewrite that unit test

AI Assistance Disclosure

AI tools were used for:

• Code generation (e.g., when writing an implementation or fixing a bug)
• Test/benchmark generation
• Documentation (including examples)
• Research and understanding
• No AI tools were used

Regardless of AI use, the human contributor remains fully responsible for correctness, design choices, licensing compatibility, and long-term maintainability.

K

[alasdairwilson]

Once more I find myself in a nabil PR that is fixing the thing I came to ndcube to fix, though in my case the issue with this was not ram usage but thread safety because copying the extracoords means deep copying the entire parent cube via this stale _ndcube which means it iterates the parent's dict while another thread is writing to it.

I think this could be more generally, i.e. not just inside _new_instance, and in a more human readable and still have equivalent fix if we patched it in a less obtuse way by simply making the deepcopy not copy the parent cube at all _ndcube from the deepcopy method on both extracoords and globalcoords...

    def __deepcopy__(self, memo):
        new = copy.copy(self)
        for key, value in vars(self).items():
            if key != "_ndcube":
                setattr(new, key, copy.deepcopy(value, memo))
        return new

or i guess you are supposed to work on the memo or something with memo[id(self)] = new first?

then no matter who or what calls deepcopy its threading safe and not memory inefficient

[Cadair]

+1 for making sure any deepcopy of a NDCube works without copying the parent cube etc

[Cadair]

Thanks, I hate it.