Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

make dockerfile more configurable #254

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

shivnagarajan
Copy link

@shivnagarajan shivnagarajan commented May 24, 2018

馃憢 Am new here 馃樃

Was trying to build containers for mcrouter using the existing Dockerfile.

Since we ideally want to build from a release version, and potentially build for either ubuntu 14.04 or 16.04 (the supported options in the Dockerfile), these changes just make the Dockerfile a little more configurable

Specifically you can call out to docker build like this.

for Ubuntu 16.04

docker build --build-arg UBUNTU_VERSION=16.04 --build-arg RELEASE_REV=release-37-0

and

for Ubuntu 14.04

docker build --build-arg UBUNTU_VERSION=14.04 --build-arg RELEASE_REV=release-37-0

The adjustable variables are

UBUNTU_VERSION with a default of 14.04
RELEASE_REV with a default of master

Also just doing

docker build with the existing Dockerfile and no arguments, will build with the default
ubuntu:14.04 and master branch (the current default)

@andreazevedo (or is there another person I could ask to review?)

@facebook-github-bot
Copy link
Contributor

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

RUN apt-get update && apt-get install -y git && \
ARG UBUNTU_VERSION
ARG RELEASE_REV=master
RUN apt-get update && apt-get install -y software-properties-common git sudo && \
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

14.04 needs software-properties-common, and 16.04 needs sudo

ARG UBUNTU_VERSION
ARG RELEASE_REV=master
RUN apt-get update && apt-get install -y software-properties-common git sudo && \
add-apt-repository ppa:george-edison55/cmake-3.x && \
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed for cmake >= 3.0

./install_ubuntu_14.04.sh $MCROUTER_DIR && \
./clean_ubuntu_14.04.sh $MCROUTER_DIR && rm -rf $MCROUTER_DIR/repo && \
./install_ubuntu_${UBUNTU_VERSION}.sh $MCROUTER_DIR && \
{ ./clean_ubuntu_${UBUNTU_VERSION}.sh $MCROUTER_DIR || true ; } && rm -rf $MCROUTER_DIR/repo && \
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no clean script for 16.04, but we can just try to call out to clean and accept and move on if it does not exist or fails.

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

@shivnagarajan shivnagarajan force-pushed the make_dockerfile_more_configurable branch from f4a8dbf to e2e6aa8 Compare May 25, 2018 02:16
@shivnagarajan
Copy link
Author

@andreazevedo not sure if you are the right person to ping on this.

The change to the Dockerfile would help us build mcrouter into containers using appropriately tagged release version and also selectively for different versions of either ubuntu 14.04 or 16.04

Thanks in advance.

@hzluyang
Copy link

hello
when I use this dockerfile to build mcrouter images
it fails

come across this error below:
libtool: link: g++-5 -std=gnu++14 -DLIBMC_FBTRACE_DISABLE -DDISABLE_COMPRESSION -Wno-missing-field-initializers -Wno-deprecated -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o mock_mc_server test/mock_mc_server-MockMc.o test/mock_mc_server-MockMcServer.o -pthread -L/usr/local/mcrouter/install/lib -ljemalloc ../../lib/libmcrouter.a -lwangle /usr/local/mcrouter/install/lib/libfolly.so -ldl -ldouble-conversion -lz -lssl -lcrypto -levent -lgflags -lglog -L/usr/lib/x86_64-linux-gnu -lboost_context -lboost_filesystem -lboost_program_options -lboost_system -lboost_regex -lboost_thread -lpthread -pthread -Wl,-rpath -Wl,/usr/local/mcrouter/install/lib -Wl,-rpath -Wl,/usr/local/mcrouter/install/lib
/usr/local/mcrouter/install/lib/libfolly.so: undefined reference to folly::initLoggingOrDie(folly::Range<char const*>)' collect2: error: ld returned 1 exit status make[4]: *** [mock_mc_server] Error 1 make[4]: Leaving directory /usr/local/mcrouter/repo/mcrouter/mcrouter/lib/network'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory /usr/local/mcrouter/repo/mcrouter/mcrouter/lib/network' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory /usr/local/mcrouter/repo/mcrouter/mcrouter/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/local/mcrouter/repo/mcrouter/mcrouter'
make: *** [all] Error 2
The command '/bin/sh -c apt-get update && apt-get install -y software-properties-common git sudo && mkdir -p $MCROUTER_DIR/repo && cd $MCROUTER_DIR/repo && git clone $MCROUTER_REPO && cd $MCROUTER_DIR/repo/mcrouter/mcrouter/scripts &amp;&amp; ./install_ubuntu_${UBUNTU_VERSION}.sh $MCROUTER_DIR &amp;&amp; { ./clean_ubuntu_${UBUNTU_VERSION}.sh $MCROUTER_DIR || true ; }&& rm -rf $MCROUTER_DIR/repo && ln -s $MCROUTER_DIR/install/bin/mcrouter /usr/local/bin/mcrouter' returned a non-zero code: 2

it looks like
/usr/local/mcrouter/install/lib/libfolly.so: undefined reference to `folly::initLoggingOrDie(folly::Range<char const*>)'

I use ubuntu:14.04

did you ever see this error before and how to fix it

@shivnagarajan
Copy link
Author

shivnagarajan commented May 30, 2018

hello
when I use this dockerfile to build mcrouter images
it fails

@hzluyang I am able to build using a released version without any errors (based on #251 (comment), that is also a recommended approach?

docker build --build-arg UBUNTU_VERSION=14.04 --build-arg RELEASE_REV=release-37-0 mcrouter/scripts/docker

this builds everything,

...
...
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/mcrouter/install/lib
...
...
+ autoreconf --install
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-aux'.
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
configure.ac:26: installing 'build-aux/compile'
configure.ac:26: installing 'build-aux/config.guess'
configure.ac:26: installing 'build-aux/config.sub'
configure.ac:21: installing 'build-aux/install-sh'
configure.ac:21: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
+ LD_LIBRARY_PATH=/usr/local/mcrouter/install/lib:
+ LD_RUN_PATH=/usr/local/mcrouter/install/lib:
+ LDFLAGS='-L/usr/local/mcrouter/install/lib -ljemalloc '
+ CPPFLAGS='-I/usr/local/mcrouter/install/include '
+ ./configure --prefix=/usr/local/mcrouter/install
...
...
...
mv -f .deps/mcpiper-Util.Tpo .deps/mcpiper-Util.Po
/bin/bash ../../libtool  --tag=CXX   --mode=link g++-5 -std=gnu++14  -DLIBMC_FBTRACE_DISABLE -DDISABLE_COMPRESSION  -Wno-missing-field-initializers -Wno-deprecated -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2  -L/usr/local/mcrouter/install/lib -ljemalloc  -o mcpiper mcpiper-AnsiColorCodeStream.o mcpiper-Config.o mcpiper-FifoReader.o mcpiper-main.o mcpiper-McPiper.o mcpiper-MessagePrinter.o mcpiper-StyledString.o mcpiper-Util.o ../../lib/libmcrouter.a -lwangle -lfolly -ldouble-conversion -lz -lssl -lcrypto -levent -lgflags -lglog  -L/usr/lib/x86_64-linux-gnu -lboost_context -lboost_filesystem       -lboost_program_options -lboost_system -lboost_regex       -lboost_thread -lpthread -pthread
libtool: link: g++-5 -std=gnu++14 -DLIBMC_FBTRACE_DISABLE -DDISABLE_COMPRESSION -Wno-missing-field-initializers -Wno-deprecated -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o mcpiper mcpiper-AnsiColorCodeStream.o mcpiper-Config.o mcpiper-FifoReader.o mcpiper-main.o mcpiper-McPiper.o mcpiper-MessagePrinter.o mcpiper-StyledString.o mcpiper-Util.o -pthread  -L/usr/local/mcrouter/install/lib -ljemalloc ../../lib/libmcrouter.a -lwangle /usr/local/mcrouter/install/lib/libfolly.so -ldouble-conversion -lz -lssl -lcrypto -levent -lgflags -lglog -L/usr/lib/x86_64-linux-gnu -lboost_context -lboost_filesystem -lboost_program_options -lboost_system -lboost_regex -lboost_thread -lpthread -pthread -Wl,-rpath -Wl,/usr/local/mcrouter/install/lib -Wl,-rpath -Wl,/usr/local/mcrouter/install/lib
...
...
...
+ printf '%s\n' 'Mcrouter installed in /usr/local/mcrouter/install/bin/mcrouter'
Mcrouter installed in /usr/local/mcrouter/install/bin/mcrouter
+ sudo apt-get install -y libdouble-conversion1 libgflags2 libboost-program-options1.54.0 libboost-filesystem1.54.0 libboost-system1.54.0 libboost-regex1.54.0 libboost-thread1.54.0 libboost-context1.54.0 libgoogle-glog0 libevent-2.0-5 libjemalloc1
...
...
...
+ sudo apt-get autoclean -y
Reading package lists...
Building dependency tree...
Reading state information...
+ sudo apt-get clean -y
+ [[ x/usr/local/mcrouter != \x ]]
+ PKG_DIR=/usr/local/mcrouter/pkgs
+ INSTALL_DIR=/usr/local/mcrouter/install
+ strip /usr/local/mcrouter/install/bin/mcrouter
+ rm -rf /usr/local/mcrouter/pkgs
+ rm -rf /usr/local/mcrouter/install/lib/libfolly.a /usr/local/mcrouter/install/lib/libfollybenchmark.a /usr/local/mcrouter/install/lib/libfollyinit.a /usr/local/mcrouter/install/lib/libfollylogging.a /usr/local/mcrouter/install/lib/libwangle.a
+ rm -rf /usr/local/mcrouter/install/include
Removing intermediate container 3262939fe709
 ---> de23b1e9e896
Step 10/10 : ENV             DEBIAN_FRONTEND newt
 ---> Running in cd7f2d363286
Removing intermediate container cd7f2d363286
 ---> c38d9799da2d
Successfully built c38d9799da2d

I guess part of the benefit of this Dockerfile change is to allow building containers from released versions.

Hope this helps.

@shivnagarajan shivnagarajan force-pushed the make_dockerfile_more_configurable branch from e2e6aa8 to ebe5f43 Compare May 31, 2018 15:20
@shivnagarajan shivnagarajan force-pushed the make_dockerfile_more_configurable branch from ebe5f43 to 956e37b Compare June 19, 2018 12:32
@shivnagarajan
Copy link
Author

Could #241 be integrated into this PR also?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants