I have a class method which I would like to use with multiprocessing.Pool for parallelisation. As class instances are not pickleable, I have used the following:
import copy_reg
import types
def _reduce_method(m):
if m.im_self is None:
return getattr, (m.im_class, m.im_func.func_name)
else:
return getattr, (m.im_self, m.im_func.func_name)
copy_reg.pickle(types.MethodType, _reduce_method)
This works no problem. However, within my class I use the GDAL module (https://pypi.org/project/GDAL/) for manipulating geospatial images and data. So now, I get the following error:
cPickle.PicklingError: Can't pickle <type 'SwigPyObject'>: attribute lookup __builtin__.SwigPyObject failed
I'm using Python 2.7.10. I know that I could solve my Pool problem using Python 3, or by using Pathos instead of Multiprocessing, but I can't do either of these easily because of network restrictions on my machine.
I've spent a bit of time searching for a solution, but to no avail. I've found some potential solutions (e.g. How to make my SWIG extension module work with Pickle?) but I'm not sure how to implement them, as I do not deliberately create a SWIG object in my code, but it must come in at some point due to GDAL.
Is there a way of registering type 'SwigPyObject' as pickleable using copy_reg, as I did with type 'instancemethod' above?
