[gcc] g++-10 does not link to the correct libgfortran library

[MarkWieczorek]

I am having problems using homebrew's g++-10 to link to the gfortran library. I suspect that g++-10 is not searching in the correct place to find the homebrew version of this library, or that there is a probably with the library search path.

Here is the problem:

1. If I don't include "-lgfortran" when linking my code, I get a lot of obvious link errors.
2. If I include "-lgfortran", I get fewer errors, but the code doesn't compile.
3. If when linking I manually specify the library as "-L /usr/local/lib/gcc/10/ -lgfortran" everything works fine!

This seems to suggest to me that g++-10 is not linking to the correct gfortran library in (2) above.

If tried to find out which libraries g++-10 looks for with the command

g++-10 -print-search-dirs
install: /usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/
programs: =/usr/local/Cellar/gcc/10.2.0/libexec/gcc/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/libexec/gcc/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/libexec/gcc/x86_64-apple-darwin19/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../../../../x86_64-apple-darwin19/bin/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../../../../x86_64-apple-darwin19/bin/
libraries: =/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../../../../x86_64-apple-darwin19/lib/x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../../../../x86_64-apple-darwin19/lib/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../x86_64-apple-darwin19/10.2.0/:/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0/../../../

Here I note that the last library path does in fact point to the correct gfortran library. However, I also note that several of the paths before this do not exist. This is just a guess, but it is possible that the search path is corrupted by these missing folders, and g++-10 stops searching the library path as soon as a non-existent folder is found.

[fxcoudert]

Without having a detailed report, it is hard to guess what is happening (“I get fewer errors” is not enough information).

Item 1 is the expected error. Item 3 is working fine. Item 2 should work, and in fact is working fine for me:

rmeur ~ $ cat a.f90 
subroutine foo() bind(c)
  print *, "Hello world"
end subroutine
rmeur ~ $ cat b.cpp 
extern "C" { void foo(void); }
int main (void) { foo(); }
rmeur ~ $ gfortran-10 -c a.f90       
rmeur ~ $ g++-10 a.o b.cpp -lgfortran
rmeur ~ $ ./a.out 
 Hello world
rmeur ~ $ otool -L ./a.out
./a.out:
	/usr/local/opt/gcc/lib/gcc/10/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/usr/local/opt/gcc/lib/gcc/10/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.28.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	/usr/local/lib/gcc/10/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

[MarkWieczorek]

Thanks for the quick response. First, your example above works fine for me. However, with my more complicated code, I am still having a problem where -L /usr/local/lib/gcc/10/ -lgfortran works, but -lgfortran doesn't.

Here is a more detailed report (simplified as much as possible):

Using -lgfortran

g++-10  -I /Users/me/src -c -o test.o test.cpp
g++-10 -o test test.o -lgfortran -L /Users/me/lib -lSHTOOLS -L/usr/local/lib -lfftw3 -lm -llapack -lblas
Undefined symbols for architecture x86_64:
  "__ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4sizeEv", referenced from:
      _main in test.o
  "__ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5c_strEv", referenced from:
      _main in test.o
  "__ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_", referenced from:
      _main in test.o
  "__ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev", referenced from:
      _main in test.o
  "__ZdlPvm", referenced from:
      __ZN9__gnu_cxx13new_allocatorIdE10deallocateEPdm in test.o
ld: symbol(s) not found for architecture x86_64

Using -L /usr/local/lib/gcc/10/ -lgfortran

g++-10  -I /Users/me/src -c -o test.o test.cpp
g++-10 -o test test.o -L /usr/local/lib/gcc/10/ -lgfortran -L /Users/me/lib -lSHTOOLS -L/usr/local/lib -lfftw3 -lm -llapack -lblas

--> Make of cpp test suite successful

[gromgit]

Since:

• a test that only involves libgfortran works without -L/usr/local/lib/gcc/10
• one that also involves other libraries can't build without it
• the missing symbols are all related to libstdc++

it's more likely that one or more of those other libraries was compiled with an earlier version of GCC, which probably isn't installed on your system, so the rpaths baked into them simply don't exist.

In that case, you can either rebuild those offending libraries with GCC 10, or stick with -L/usr/local/lib/gcc/10 in your build.

[MarkWieczorek]

Thanks for the help, but I still don't understand this issue: When I force verbose output when linking my code, it provides the following library search path:

Library search paths:
	/Users/lunokhod/SphericalHarmonics/shtools-git/lib
	/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0
	/usr/local/lib
	/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin19/10.2.0
	/usr/local/Cellar/gcc/10.2.0/lib/gcc/10
	/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/lib

/usr/local/Cellar/gcc/10.2.0/lib/gcc/10 is the same thing as /usr/local/lib/gcc/10 specified in the previous example. So, why do I need to specify -L /usr/local/Cellar/gcc/10.2.0/lib/gcc/10 when linking my code when it is already in the linker search path? Isn't this redundant? (Obviously is not, because I need to do so.)

[fxcoudert]

Without having a detailed report, it is hard to guess what is happening

Right now, we can only guess. My attempt to reproduce your minimal test case is passing, so obviously there is something more to it. @gromgit has a hypothesis about what is happening, which is as good as anything else I could guess. Without a fully reproducible report, we cannot know.

Also, when I encounter difficult bugs, I can solve them myself in the process of simplification to a reproducible example. At some point in the simplification, the bug will disappear and then you will have characterised the crucial element that is triggering it.

[MarkWieczorek]

One more piece of information regarding this problem:

If I use Apple's default g++ compiler and then link to homebrew's gfortran library, everything works fine. It is only when I use homebrew's g++-10 (and -9) compiler that the above problems arise.

I understand that I have not provided a simple code for you to reproduce this problem (I am working with a large and complicated code base). Nevertheless, it seems pretty clear that homebrew's g++ does not properly search for -lgfortran in /usr/local/Cellar/gcc/10.2.0/lib/gcc/10.

[fxcoudert]

Nevertheless, it seems pretty clear that homebrew's g++ does not properly search for -lgfortran in /usr/local/Cellar/gcc/10.2.0/lib/gcc/10

Except it does, as I have shown on a simple example in #11 (comment)

You have also shown that g++-10 -print-search-dirs does include the right directory. Also, the hypothesis that “the search path is corrupted by these missing folders” is not valid: missing directories in the search path are silently ignored.

In short: the origin of the bug is unclear, and we have currently no way to reproduce it. So it is unclear how you think we can investigate further.

[fxcoudert]

I'll also add that something is very weird: the Undefined symbols errors you listed above are not related to Fortran, but to C++. And it's very weird for g++ not to find its own library… unless there is a mixture of code compiled with different compilers or compiler versions (as @gromgit stated earlier).

[gromgit]

@MarkWieczorek , your test only uses the following libraries (full paths guessed from your -L flags):

• /Users/me/lib/libSHTOOLS.dylib
• /usr/local/lib/libfftw3.dylib
• /usr/local/lib/liblapack.dylib
• /usr/local/lib/libblas.dylib

Run otool -L on each of the above libraries and post the output.