Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

protobuf link error when run test on Macos #2457

Closed
dzhwinter opened this issue Jun 13, 2017 · 7 comments
Closed

protobuf link error when run test on Macos #2457

dzhwinter opened this issue Jun 13, 2017 · 7 comments

Comments

@dzhwinter
Copy link
Contributor

build with the lastest develop branch run test will report me protobuf link error. I can't run any test case. Error message e.g.

Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 29
    Start 29: test_ActivationGrad

29: Test command: /Users/dzh/github/PaddleDevelop/Paddle/build/paddle/gserver/tests/test_ActivationGrad
29: Test timeout computed to be: 9.99988e+06
29: dyld: Symbol not found: __ZNK6google8protobuf7Message13SpaceUsedLongEv
29:   Referenced from: /Users/dzh/github/PaddleDevelop/Paddle/build/paddle/gserver/tests/test_ActivationGrad
29:   Expected in: flat namespace
29:  in /Users/dzh/github/PaddleDevelop/Paddle/build/paddle/gserver/tests/test_ActivationGrad
1/1 Test #29: test_ActivationGrad ..............***Exception: Other  0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.02 sec

The following tests FAILED:
         29 - test_ActivationGrad (OTHER_FAULT)
Errors while running CTest

paddle use the thrid_party protobuf.

-- Found Paddle host system: macosx
-- Found Paddle host system's CPU: 4 cores
-- Protobuf protoc executable: /usr/local/bin/protoc
-- Protobuf library: /Users/dzh/github/PaddleDevelop/Paddle/third_party/install/protobuf/lib/libprotobuf.a
-- Protobuf version: 3.3
-- BLAS library: /Users/dzh/github/PaddleDevelop/Paddle/third_party/install/openblas/lib/libopenblas.a
-- Ccache is founded, use ccache to speed up compile.
-- Paddle version is 0.10.0
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/dzh/github/PaddleDevelop/Paddle/build
@Xreki
Copy link
Contributor

Xreki commented Jun 14, 2017

-- Protobuf protoc executable: /usr/local/bin/protoc
-- Protobuf library: /Users/dzh/github/PaddleDevelop/Paddle/third_party/install/protobuf/lib/libprotobuf.a
-- Protobuf version: 3.3

It is strange. The found protoc executable is from the standard path, of which the version is 3.3, but the protobuf library is from the third_party, which is built and installed by PaddlePaddle and the version is 3.1. I guess the error is caused by different version between protoc and libprotobuf.a. Maybe there is protoc in /usr/local/bin, but no libprotobuf.a in /usr/local/lib? Could you please check it?

@dzhwinter
Copy link
Contributor Author

Yes, It is a mismatch problem of protobuf. Here is my environment.
system wide protobuf is 3.3.0

/usr/local/bin/protoc
/usr/local/lib/libprotobuf.a
// typed config command it shows 
 pkg-config --libs protobuf
-L/usr/local/Cellar/protobuf/3.3.0/lib -lprotobuf -D_THREAD_SAFE

I had made a double check, everything looks correct in the right place.
The third_party version is protobuf 3.1.0

/Users/dzh/github/PaddleDevelop/Paddle/third_party/install/protobuf/bin/protoc
/Users/dzh/github/PaddleDevelop/Paddle/third_party/install/protobuf/lib/libprotobuf.a

seems nothing is wrong.

So, my problem is why paddle choose binary in system and the library from third_party?, I thought the answer will show us the right way.

Thanks!

@Xreki
Copy link
Contributor

Xreki commented Jun 14, 2017

@dzhwinter What is your cmake command?

@dzhwinter
Copy link
Contributor Author

only one command of cmake .., nothing else

@typhoonzero
Copy link
Contributor

@dzhwinter you can add some debug logs to see what exactly happend:

SET(PROTOBUF_VERSION 3.1)
IF(NOT CMAKE_CROSSCOMPILING)
    FIND_PACKAGE(Protobuf ${PROTOBUF_VERSION})
    message(STATUS ${PROTOBUF_ROOT})
    IF(PROTOBUF_FOUND)
        message(STATUS "found system protobuf")
        SET_PROTOBUF_VERSION()
        IF("${PROTOBUF_VERSION}" VERSION_LESS "3.1.0")
            SET(PROTOBUF_FOUND OFF)
        ENDIF()
    ENDIF(PROTOBUF_FOUND)
ELSE()
    build_protobuf(protobuf_host TRUE)
    LIST(APPEND external_project_dependencies protobuf_host)

    SET(PROTOBUF_PROTOC_EXECUTABLE ${protobuf_host_PROTOC_EXECUTABLE}
        CACHE FILEPATH "protobuf executable." FORCE)
ENDIF()

IF(NOT PROTOBUF_FOUND)
    message("need to build protobuf")
    build_protobuf(protobuf FALSE)
    LIST(APPEND external_project_dependencies protobuf)

    SET(PROTOBUF_INCLUDE_DIR ${protobuf_INCLUDE_DIR}
        CACHE PATH "protobuf include directory." FORCE)
    IF(NOT CMAKE_CROSSCOMPILING)
        SET(PROTOBUF_PROTOC_EXECUTABLE ${protobuf_PROTOC_EXECUTABLE}
            CACHE FILEPATH "protobuf executable." FORCE)
    ENDIF()
    SET(PROTOBUF_LITE_LIBRARY ${protobuf_LITE_LIBRARY} CACHE FILEPATH "protobuf lite library." FORCE)
    SET(PROTOBUF_LIBRARY ${protobuf_LIBRARY} CACHE FILEPATH "protobuf library." FORCE)
    SET(PROTOBUF_PROTOC_LIBRARY ${protobuf_PROTOC_LIBRARY} CACHE FILEPATH "protoc library." FORCE)
ENDIF(NOT PROTOBUF_FOUND)

@dzhwinter
Copy link
Contributor Author

after delete all these mess of cmake code, just leave behind of

INCLUDE(ExternalProject)
FIND_PACKAGE(Protobuf ${PROTOBUF_VERSION})
INCLUDE_DIRECTORIES(${PROTOBUF_INCLUDE_DIR})

use system find_package built-in function, everything works well. Still have no idea why that problem happened.

@typhoonzero
Copy link
Contributor

If you can reproduce this issue, please record the exact step, and reopen this issue. Thanks.

heavengate added a commit to heavengate/Paddle that referenced this issue Aug 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants