I've found a library which is in C (?), that I want to try from Python: https://github.com/centaurean/density
I'd like to see if I can store compressed files on disk and decompress them in memory for faster full-file reading times.
I'm new to using external code from Python. How can I create a Python function that uses this library with a little overhead as possible?
(I will work with Windows or Linux)
One way is to compile the library into a dynamic library (.dll on Windows or .so on Linux) and use ctypes (https://docs.python.org/2/library/ctypes.html) to access it.
ctypes provides an easy path into doing exactly this through FFI, with the alternative of using something like SWIG to interface with the python libraries at a much lower layer.
I would recommend generating a little micro-benchmark program to record what the performance is before and after such an experiment, so that you have concrete values to use in choosing if you want to run with ctypes or python-swig code long term over doing something natively in python, which you could trial in c-python or a variant (such as pypy)
Fortunately for you, Python comes with an FFI built-in called ctypes. It works with both Linux and Windows, and there are plenty of tutorials out there to learn how to use it.
Note that you will need to compile the library beforehand into a shared lib (DLL in Windows, SO in Linux).
