component/python-2.7: use slapos's gcc libraries.

component/python-2.7/buildout.cfg

@@ -13,6 +13,7 @@ extends =
   ../zlib/buildout.cfg
   ../file/buildout.cfg
   ../xz-utils/buildout.cfg
+  ../gcc/buildout.cfg
 
 parts =
     python2.7
@@ -60,4 +61,4 @@ make-targets = make profile-opt && make install
 environment =
   PATH=${patch:location}/bin:${xz-utils:location}/bin:%(PATH)s
   CPPFLAGS=-I${zlib:location}/include -I${readline:location}/include -I${libexpat:location}/include -I${libffi:location}/include -I${ncurses:location}/include -I${ncurses:location}/include/ncursesw -I${bzip2:location}/include  -I${gdbm:location}/include -I${openssl:location}/include -I${sqlite3:location}/include -I${gettext:location}/include
-  LDFLAGS=-L${zlib:location}/lib -L${readline:location}/lib -L${libexpat:location}/lib -L${libffi:location}/lib -L${ncurses:location}/lib -L${bzip2:location}/lib -L${gdbm:location}/lib -L${openssl:location}/lib -L${sqlite3:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${readline:location}/lib -Wl,-rpath=${libexpat:location}/lib -Wl,-rpath=${libffi:location}/lib -Wl,-rpath=${ncurses:location}/lib -Wl,-rpath=${bzip2:location}/lib -Wl,-rpath=${gdbm:location}/lib -Wl,-rpath=${openssl:location}/lib -Wl,-rpath=${sqlite3:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -Wl,-rpath=${file:location}/lib
+  LDFLAGS=-L${zlib:location}/lib -L${readline:location}/lib -L${libexpat:location}/lib -L${libffi:location}/lib -L${ncurses:location}/lib -L${bzip2:location}/lib -L${gdbm:location}/lib -L${openssl:location}/lib -L${sqlite3:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${readline:location}/lib -Wl,-rpath=${libexpat:location}/lib -Wl,-rpath=${libffi:location}/lib -Wl,-rpath=${ncurses:location}/lib -Wl,-rpath=${bzip2:location}/lib -Wl,-rpath=${gdbm:location}/lib -Wl,-rpath=${openssl:location}/lib -Wl,-rpath=${sqlite3:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -Wl,-rpath=${file:location}/lib -Wl,-rpath=${gcc:location}/lib64

[Kazuhiko Shiozaki]

May I ask you why ? (and please write 'why' in commit log).

• For 32-bit architecture case, please add ${gcc:location}/lib as well.
• Now rebootstrap also builds gcc...

[Rafael Monnerat]

I'm also curious...

[Kazuhiko Shiozaki]

Also why do you want to link with slapos's gcc library but not compile with slapos's gcc ?

[Yusei Tahara]

Sorry, I did not explain. It is needed for tensorflow. Tensorflow uses an unique building system named bazel. Bazel is similar to our slapos buildout approach, it aims to builds everything. But so far, bazel depends on many binaries provided by linux distribution and bazel can't allow user to use local binaries(bazel can uses /usr/lib/xxx but can't use /opt/slapgrid/xxx) and it is quite hard to change this behavior. You can find many problems like this https://github.com/bazelbuild/bazel/issues/649. I tried to tell bazel to use our local gcc libraries, but I failed. Then when I used debian 7 environment to build tensorflow by slapos, I found GLIBXXX undefined reference error, because debian 7's glib was old. I did not have this error on debian 8 but by chance my webrunner was debian 7, I found this problem. And a solution I found was this change. Recently new version of bazel was release. This problem might be solved.

[Kazuhiko Shiozaki]

Anyway this change is not acceptable as it is. We need to find a better way.

What I can propose is modifying slapos.recipe.build so that script part is invoked in another process of python with modified environment variables (like LD_LIBRARY_PATH). Please check how script is called.

[Rafael Monnerat]

I think, @yusei could move the change to bazel component and leave it there instead place it on general (as bazel that contains the bug).

[Yusei Tahara]

It is a good idea. I will move it to bazel/buildout.cfg.

[Kazuhiko Shiozaki]

With buildout2, the story is not so simple.

• if you don't use slapos.recipe.rebootstrap, the upper python is used, thus 'moving such conf to bazel/buildout.cfg' will not work
• if you use slapos.recipe.rebootstrap, gcc is built twice.

So I really ask you to find a better way.

Also please never include such an influential change directly in master without review...

[Yusei Tahara]

Sorry, next time I will ask review. And about solution, I will check the latest bazel and find a way to make bazel use slapos's gcc when I have time.

[Kazuhiko Shiozaki]

I tried to reproduce your issue on Debian 7 webrunner without touching python-2.7 but could not. Indeed, ldd parts/bazel/bin complains but building bazel itself succeeded. Can you please give me the detail of the issue you had ?

• building bazel itself failed ?
• building tensorflow using bazel failes ?

FYI, here is the output of ldd bazel in my Debian 7 webrunner.

$ ldd parts/bazel/bin/bazel                                                                     
parts/bazel/bin/bazel: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not fo
und (required by parts/bazel/bin/bazel)                                                         
parts/bazel/bin/bazel: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.8' not foun
...

[Yusei Tahara]

I think that building tensorflow succeeded but when imported it raised GLIBCXX error at _pywrap_tensorflow.so.

[Kazuhiko Shiozaki]

Then this trick should solve the issue.

[Yusei Tahara]

Oh I see. I will try it when I have time. Thanks.