[MXNET-1251] Basic configuration to do static-linking

[lanking520]

Description

For Ubuntu 14.04 base build to install all dependencies.
@szha @zachgk

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage:
• Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
• Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
• Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
• Code is well-documented:
• For user-facing API changes, API doc string has been updated.
• For new C++ functions in header files, their functionalities and arguments are documented.
• For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
• Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
• To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

[marcoabreu]

Any specific reason we aim for Ubuntu 14? Ubuntu 18 has already been released

[lanking520]

@marcoabreu As far as I know. Cent OS 7, Amazon Linux all share the same GLIBC version with Ubuntu 14.04 and Ubuntu 16.04 has a higher GLIBC version. To make it compatible, we should use 14.04 as the base OS

[zachgk]

I recommend using the maven3 ppa (sudo add-apt-repository ppa:andrei-pozolotin/maven3) and combining all of the apt-get installs into one call. Also just run one apt-get update after all of the ppa additions and use "set -ex"

[marcoabreu]

Yeah but then we have to maintain two systems. RHEL is known to always be behind due to their strict support, but that shouldn't limit us for support of other operating systems. Would be great if you could separate that

[marcoabreu]

Please split RHEL and Debian based pipelines

[lanking520]

@marcoabreu This is the publish configuration that works for all different platforms and this will not be used in all the rest CIs for testing. The optimal goal is using one configuration to create a pip or maven package that works for all platforms. We do have a plan to upgrade this into 16.04 once we have a test pipeline set up in Jenkins. This script is just to inherit the legacy of current publish pipeline.

About the RHEL and Debian diff. Firstly the published stuff is all the same no matter on RHEL or Debian and can be run on either. Introducing a different publish pipeline for these two in this case is not necessary as it introduce more maintainance nightmare.

[marcoabreu]

Thanks for the insight, Qing! Can the binaries you create with this script in that form be loaded into ubuntu16/18 and other systems that have later versions? If yes, then I'm fine with that approach :)

[lanking520]

Thanks for the insight, Qing! Can the binaries you create with this script in that form be loaded into ubuntu16/18 and other systems that have later versions? If yes, then I'm fine with that approach :)

The answer is yes. In fact, @frankfliu has already working on producing the Dockerfile from this script that works for 16.04 with the publish pipeline. However it depends on the setup of the test-publish step to be ready and then we can test them through the pipeline instead of dive into different OS VMs to test manually.

[marcoabreu]

Alright thanks for your explanation. Please feel free to go ahead then.

You might want to add a comment in the code which packages specifically have to stay at a certain version and that they are chosen as the smallest common denominator

[roywei]

@mxnet-label-bot add[Build, pr-awaiting-merge]