How to properly optimize memory management when designing a custom library in python?

Question

I'm creating a library in Python 2.7. Since I'm going to be calling this library (and others like it) like this many, many times, I need to make sure I'm not creating a bunch of objects (like gm below) which will end up wasting memory when they are no longer needed.

However, I'm a bit confused about the fact that with my code, the gm object still exists outside of the with:

import mylib
with mylib.GeneralMethods() as gm:
    print gm.say_hello()
>>> hello world

print gm # this is no longer needed, but still exists...
>>> <mylib.GeneralMethods object at 0x10f9c5bd0>

Is there anything I can do to further optimize memory management when designing and calling my library? I don't want to have stray objects eating memory, like gm.

Right now, this is what my library looks like (heavily simplified):

class GeneralMethods(object):
    def __init__(self, parent=None):
        super(GeneralMethods, self).__init__()

    def __enter__(self):
        return self

    def __exit__(self, type, value, traceback):
        return isinstance(value, TypeError)

    def say_hello(self):
        return 'hello world'

In CPython, typically objects are collected by GC as soon as name goes out of scope (although in general it's not guaranteed that it'll be collected immediately). In your code snippet your context manager (name `gm` still exists in your scope). Why Python should assume it could be safely deleted? — Łukasz Rogalski, Jan 03 '16 at 12:20
`gm` is just another variable, it's not garbage collected before execution leaves the scope in which it was defined. — GingerPlusPlus, Jan 03 '16 at 12:48
@GingerPlusPlus Ah, I see. Then I'm doing it all wrong, basically, as I misunderstood how `with` works. :) — fredrik, Jan 03 '16 at 14:56
The optimal way is not to worry about it unless you have reason to do so. Start worrying if your library forces the OS to start swapping to disk... — Roland Smith, Jan 03 '16 at 17:57

score 1 · Accepted Answer · edited May 23 '17 at 12:15

I'm a bit confused about the fact that with my code, the gm object still exists outside of the with

That's expected behavior; gm is just another variable, it's not garbage collected before execution leaves the scope in which it was defined.

If you are sure that it's worth extra line, you can do it explicitly:

del gm

Note that it does not guarantee that the object will be instantly garbage collected – it just makes it eligible for garbage collection before the execution leaves the scope, if gm is the only reference to the object.

If you're concerned about memory, take a look at __slots__.

How to properly optimize memory management when designing a custom library in python?

1 Answers1