在 C 中传递 ctypes 回调函数会导致内存相关问题

编程问答 2022-05-28

问题描述

我在使用 ctypes 移植到 Python 的某些 C 代码中遇到了一个稍微复杂的错误。我已经能够在一个最小的工作示例中重现这个问题，并希望得到一些帮助。问题的本质是使用 ctypes 回调会导致内存相关问题如果调用通过指针传递给另一个 C 函数。

这里是 C 代码：

// the function pointer that we want to use as a callback from python
typedef void(*fn)(double *,double *,double *);

// a place to store the func (among other things in the real code)
typedef struct {
    fn func;
} Parameters;


// wrapper around calls to allocate instance of Parameters from Python
Parameters **makeParameters() {
    Parameters **pp;
    pp = (Parameters **) malloc(sizeof(Parameters *));
    *pp = (Parameters *) malloc(sizeof(Parameters));
    return pp;
}

// clean up memory
void freeParameters(Parameters **parameters) {
    free(*parameters);
    free(parameters);
}

// collects input parameters. Store the callback inside the Parameters struct
// for later use in other functions. 
void setup_fn(Parameters **parameters,fn callback_fn) {
    (*parameters)->func = callback_fn;
}

// Worker function that actually calls the callback
void do_work_with_func(Parameters **parameters){
    double *x = (double *) malloc(sizeof(double) * 2);
    double *y = (double *) malloc(sizeof(double));
    double *z = (double *) malloc(sizeof(double));

    x[0] = 1.0;
    x[1] = 2.0;
    *y = 3.0;
    *z = 0;

    (*parameters)->func(x,y,z);

    printf("From C: Output from do_work_with_func fn: %f",*z);
    free(x);
    free(y);
    free(z);
}

用 C 编译这个

void callback_fn(double* x,double* y,double* z){
    printf("From C: I'm a callback function\n");
    *z = x[0] + (x[1] * *y); // output 7 with test numbers.
}

int main(){
    Parameters ** params = makeParameters();
    setup_fn(params,callback_fn);
    do_work_with_func(params);
    freeParameters(params);
}

生产

From C: I'm a callback function
From C: Output from do_work_with_func fn: 7.000000
Process finished with exit code 0

一切都很好。所以现在让我们尝试使用 ctypes 库从 Python 中使用它。


# load the library
lib = ct.CDLL("library_name")

# load the functions
makeParameters = lib.makeParameters
makeParameters.argtypes = []
makeParameters.restype = ct.c_int64 # raw pointer

freeParameters = lib.freeParameters
freeParameters.argtypes = [ct.c_int64]
freeParameters.restype = None

FN_PTR = ct.CFUNCTYPE(None,ct.POINTER(ct.c_double*2),ct.POINTER(ct.c_double),ct.POINTER(ct.c_double))

setup_fn = lib.setup_fn
setup_fn.argtypes = [ct.c_int64,FN_PTR]
setup_fn.restype = None

do_work_with_func = lib.do_work_with_func
do_work_with_func.argtypes = [ct.c_int64]
do_work_with_func.restype = None

# main program 
def callback_fn(x,z):
    print("From Python: I'm a callback function")
    z.contents.value = x.contents[0] + (x.contents[1] * y.value)

params = makeParameters()
setup_fn(params,FN_PTR(callback_fn))
do_work_with_func(params)

有时会导致

OSError: [WinError -1073741795] Windows Error 0xc000001d

或有时导致

OSError: exception: access violation reading 0xFFFFFFFFFFFFFFFF

谁能解释一下这里发生了什么以及如何修复程序？预期的输出应该是

From Python: I'm a callback function
From C: Output from do_work_with_func fn: 7.000000

解决方法

setup_fn(params,FN_PTR(callback_fn)) 包装 callback_fn 将包装的对象传递给函数，然后释放 FN_PTR 对象。您必须在可以调用的生命周期内维护对象。最简单的方法是永久装饰函数：

FN_PTR = ct.CFUNCTYPE(None,ct.POINTER(ct.c_double*2),ct.POINTER(ct.c_double),ct.POINTER(ct.c_double))

@FN_PTR  # decorate here
def callback_fn(x,y,z):
    print("From Python: I'm a callback function")
    z.contents.value = x.contents[0] + (x.contents[1] * y.value)

setup_fn(params,callback_fn) # call with decorated function.
ccallbackctypeslanguage-interoperabilitypython