Embed Python3 without standard library

PythonPython 3.xEmbeddingStandard Library

Python Problem Overview


EDIT: I have asked an opposing question here: https://stackoverflow.com/questions/34724879/how-to-embed-python3-with-the-standard-library

A solution for Python2 is provided here: https://stackoverflow.com/questions/20951624/is-it-possible-to-embed-python-without-the-standard-library

However, Python3 fails on Py_Initialize(); with:

Fatal Python error: Py_Initialize: unable to load the file system  
codec ImportError: No module named 'encodings'

This makes sense because py3 source files are utf-8 by default. So it seems that it requires an external binary just in order to parse py3 source files.

So what to do?

It looks as though I need to locate the encodings binary in my system Python installation, copy this into my project tree, and maybe set some environment variable PYTHONPATH(?) So that my libpython.dylib can find it.

Is it possible to avoid this? And if not, can anyone clarify the steps I need to take? Are there going to be any more hiccups?


NOTE: For posterity, this is how I got a stand-alone libpython.dylib linking into my project on OSX:

First I locate my system Python's library: /usr/local/Frameworks/Python.framework/Versions/3.4/Python (in my case it was installed with homebrew).

Now I:

  • copy the .dylib into my project folder creating ./Libs/libpython3.4.1_OSX.dylib

  • Go into build settings -> linking and set other linker flags to -lpython3.4.1_OSX

At this point it will appear to work. However if you know try building it on a fresh OSX installation, it will fail. This is because:

$ otool -D ./libpython3.4.1_OSX.dylib 
./libpython3.4.1_OSX.dylib:
/usr/local/Frameworks/Python.framework/Versions/3.4/Python

The .dylib is still holding onto it's old location. It's really weird to me that the .dylib contains a link to its location, as anything that uses it must know where it is in order to invoke it in the first place!

We can correct this with:

$ install_name_tool -id @rpath/libpython3.4.1_OSX.dylib libpython3.4.1_OSX.dylib

But then also in our Xcode project we must:

  • Go into build phases. Add a copy files step that copies libpython3.4.1_OSX.dylib to Frameworks (that's the right place to put it).
  • Go into build settings -> linking and set runpath search paths to @executable_path/../Frameworks/libpython3.4.1_OSX.dylib

Finally I need to go into edit scheme -> run -> arguments -> environment variables and add PYTHONHOME with value ../Frameworks

I suspect that to get this working I will also need to add a PYTHONPATH

Links:
https://mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html
http://qin.laya.com/tech_coding_help/dylib_linking.html
https://github.com/conda/conda-build/issues/279#issuecomment-67241554
https://stackoverflow.com/questions/9263256/can-you-please-help-me-understand-how-mach-o-libraries-work-in-mac-os-x
http://nshipster.com/launch-arguments-and-environment-variables/

Python Solutions


Solution 1 - Python

I have attempted this and it would take more time than you want to spend to embed Python 3 without the Python library.

Some modules in the library are necessary for Python 3 to run and it would take a lot of modifications for it to work without them.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionP iView Question on Stackoverflow
Solution 1 - PythonAldenView Answer on Stackoverflow