Android builds arm arm64 and x86 #777

faisal00813 · 2018-05-21T11:00:25Z

Signed-off-by: Mohammad Abdul Sami faisal00813@gmail.com

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com> (cherry picked from commit 2d681f6) Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com> Updated READMEs

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

lodo1995 · 2018-05-21T12:16:17Z

@faisal00813 the path /sdcard/Documents/.indy_client is not portable to all devices. For example, my phone, which is a Samsung running KitKat with no sd card, does not have it. On my phone, the documents folder is /storage/emulated/0.

Signed-off-by: Mohammad Abdul Sami faisal00813@gmail.com Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com> Fixed credentials filtering bug Signed-off-by: artem.ivanov <artem.ivanov@dsr-company.com> fixed libsqlcipher-sys version in cargo.lock Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

# Conflicts: # libindy/Cargo.lock # libindy/Cargo.toml # libindy/build.rs

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

faisal00813 · 2018-06-01T08:22:41Z

@jovfer @vimmerru Can you please review this.

vimmerru · 2018-06-05T08:55:49Z

doc/android-build.md

+
+## Prerequisites
+
+- Docker


Can i build it on windows and MacOS?

Only Linux is supported as of now.

Can you add this info to per-requirements? It is better to provide info about platform that we tested.

That info is present in Known Issues segment at the bottom of the document.

vimmerru · 2018-06-05T08:58:28Z

libindy/Cargo.toml

@@ -38,17 +38,17 @@ sodium_static = []
 indy-crypto = { version = "=0.4.1", optional = true }
 int_traits = { version = "0.1.1", optional = true }
 digest = "0.6.2"
-env_logger = "0.4.2"
+env_logger = "0.5.10"


Is it updated intentionally?

No. We require 0.4.2 for android_logger

ok, make sense

vimmerru · 2018-06-05T08:58:35Z

libindy/Cargo.toml

 errno = "0.2.3"
 etcommon-rlp = "0.2.3"
 generic-array = "0.8.3"
 hex = "0.2.0"
 libc = "0.2.21"
-log = "0.3.7"
+log = "0.4.1"


Is it updated intentionally?

ok, make sense

vimmerru · 2018-06-05T08:59:41Z

libindy/Cargo.toml

 openssl = { version = "=0.9.24", optional = true }
 owning_ref = "0.3.3"
 rand = "0.3"
-rusqlite = "0.13.0"
+rusqlite = { version = "0.13.0", features=["bundled"] }


As i understand now we start static linking with sqlite on all platforms.

On Android, we don't have SQLite binaries which we provide while building. Also, I can't recall any flag which suggests that we are doing static linking with SQLite.

The problem is that we don't want to force static linking on all platforms. Only for android and iOS

- Modified to plugged/mod.rs to support arm64 Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

vimmerru · 2018-06-13T09:21:40Z

libindy/build.rs

@@ -32,4 +32,40 @@ fn main() {
            }
        }
    }
+		match &target {
+		x if x.contains("aarch64-linux-android") || x.contains("armv7-linux-androideabi") ||


Can we just replace this complex if with just "linux-android" as it seems present in all arch strings?

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

vimmerru · 2018-06-13T11:47:49Z

libindy/build_scripts/android/Dockerfile

@@ -0,0 +1,49 @@
+FROM ubuntu:16.04


I believe we need to rename it to something more obvious like android-build-env.dockerfile

vimmerru · 2018-06-13T12:11:45Z

doc/android-build.md

+
+## How to build.
+- Goto `indy-sdk/libindy/build_scripts/android` folder
+- Run `build.dependencies.locally.sh`


Run build.dependencies.locally.sh

Will it build dependencies for all supported architectures?

Yes. The script will build for x86, arm and arm64 architectures.

vimmerru · 2018-06-13T12:19:16Z

libindy/build_scripts/android/dependencies/openssl/build.sh

+mkdir -p $ARMV7_DIR
+mkdir -p $ARMx86_DIR
+
+git clone /~https://github.com/nsivraj/openssl_for_ios_and_android.git


Build must be repeatable. Se we can't use just master branch for dependencies. Use tag/release/commit

vimmerru · 2018-06-13T12:23:46Z

libindy/build_scripts/android/dependencies/sodium/build.sh

+
+if [ ! -f "libsodium-1.0.12.tar.gz" ] ; then
+    echo "Downloading libsodium-1.0.12.tar.gz"
+    wget -q wget /~https://github.com/jedisct1/libsodium/releases/download/1.0.12/libsodium-1.0.12.tar.gz


Master uses libsodium 1.0.14

vimmerru · 2018-06-13T12:30:47Z

libindy/src/utils/environment.rs


 pub struct EnvironmentUtils {}

 impl EnvironmentUtils {
    pub fn indy_home_path() -> PathBuf {
        // TODO: FIXME: Provide better handling for the unknown home path case!!!
+


This code doesn't look correct for me.

I see 2 unwraps. Will we crash if there is no EXTERNAL_STORAGE variable?

Why we create directories here? It isn't responsibility of this function

Flow also looks very non-obvious? Can you just use on match or if/if else/if else?

Libindy will now panic! if it cannot find the EXTERNAL_STORAGE .

Agree with @vimmerru. Please fix all 3 points
#777 (comment)

AxelNennker · 2018-06-13T17:45:04Z

I suggest not to use cd ../../../.. to get to the root of indy-sdk from the build scripts but to use
e.g. for arm64

#!/usr/bin/env bash

BASEDIR=$(dirname "$0")

DEPS_FOLDER=$BASEDIR/dependencies
LIBZMQ=${DEPS_FOLDER}/zmq/libzmq_arm64
OPENSSL=${DEPS_FOLDER}/openssl/openssl_arm64
LIBSODIUM=${DEPS_FOLDER}/sodium/libsodium_arm64
$BASEDIR/build.sh -f arm64 21 aarch64-linux-android android_support ${OPENSSL} ${LIBSODIUM} ${LIBZMQ}

I have send the changed script via email to @faisal00813

Another thing: Why delete the android ndk? I copied it here so that it does not get downloaded everytime.

vimmerru · 2018-06-15T10:51:46Z

It would be nice to have a script in libindy root folder that will build artifacts for all platforms.

vimmerru · 2018-06-15T10:51:57Z

You use /~https://github.com/nsivraj/openssl_for_ios_and_android to build openssl, This script isn't performed in isolated environment. Does it require any pre-requirements? Can we just use pre-built binary from the same site?

vimmerru · 2018-06-15T10:52:18Z

Build looks a bit over-complicated now. You use dockerfiles that just perform builds of one thing. It can be much faster (and easy to understand) to have one dockerfile that setups all necessary environment and set of scripts to execute inside of this dockerfile.

vimmerru · 2018-06-15T10:52:27Z

Is there a way to execute tests?

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

[WIP] updated build scripts to download and use prebuilt dependencies Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

AxelNennker · 2018-06-22T16:16:31Z

Android NDK 17 is out. Update this?

…l the architecture. Removed dependencies building scripts and instructions. Libnullpay uses same mechanism for building as libindy Build scripts downloads prebuilt binaries while building. Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

faisal00813 · 2018-06-26T11:36:06Z

@AxelNennker I don't see any immediate need for updating to NDK17. The changelog suggests deprecation of GCC and we use CLANG in our scripts.

faisal00813 · 2018-06-26T11:37:40Z

@vimmerru I have simplified the build scripts. Now a single script in root folder of each module (libindy and libnullpay) builds for all the architectures.

AxelNennker · 2018-06-26T11:40:07Z

I switched out r16 and moved to r17 and rebuild using my scripts and had no problems doing that.
/~https://github.com/AxelNennker/indy-sdk/blob/android_builds/doc/android-build.md
I think if you try to change the 16 to 17 and they still work then switch otherwise don't.

faisal00813 · 2018-06-26T11:40:34Z

Also addressed the concern regarding /~https://github.com/nsivraj/openssl_for_ios_and_android . Now we use a fork /~https://github.com/evernym/indy-android-dependencies with a tag.

…nother function Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

…ble on android. Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

jovfer

Please fix active review comments above. Right now it's only about codestyle, if required I can review build scripts structure as next round of review.

jovfer · 2018-06-27T11:07:48Z

libindy/build.rs

@@ -32,4 +32,38 @@ fn main() {
            }
        }
    }
+		match &target {


please clean-up code here. Seems like this section should be in else if for previous one (windows)

jovfer · 2018-06-27T11:10:36Z

libindy/src/utils/environment.rs


 pub struct EnvironmentUtils {}

 impl EnvironmentUtils {
    pub fn indy_home_path() -> PathBuf {
        // TODO: FIXME: Provide better handling for the unknown home path case!!!
+


Agree with @vimmerru. Please fix all 3 points
#777 (comment)

jovfer · 2018-06-27T11:11:19Z

libnullpay/Cargo.toml

@@ -6,7 +6,7 @@ build = "build.rs"

 [lib]
 name = "nullpay"
-crate-type = ["cdylib", "rlib"]
+crate-type = ["staticlib","cdylib", "rlib"]


seems like staticlib should be android-specific target

I don't know if rust supports the conditional crate-type declaration. Does it?
There are two issues related to it though, rust-lang/cargo#5367 and rust-lang/cargo#4900 .

jovfer · 2018-06-27T11:12:31Z

libnullpay/build.rs

@@ -4,19 +4,29 @@ use std::path::Path;
 fn main() {
    let target = env::var("TARGET").unwrap();
    println!("target={}", target);
+    match &target {


match construction is ugly here. please clean-up and use if else

jovfer · 2018-06-27T11:15:24Z

libindy/Cargo.toml

+android_logger = "0.5"
+[target.'cfg(any(target_os = "android", target_os = "ios"))'.dependencies]
+rusqlite = { version = "0.13.0", features=["bundled"] }


please specify common and android rusqlite dependence as neighbors to avoid partial update in the future

jovfer · 2018-06-27T12:08:16Z

@faisal00813 After auto-merge test compilation is failed, please fix it also
https://ci.evernym.com/job/Indy%20SDK%20CI/view/change-requests/job/PR-777/8/execution/node/97/log/

into android_builds

updated android-build.doc Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

…cargo.toml for better visibility Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

faisal00813 force-pushed the android_builds branch from 8de78a4 to 686e877 Compare May 21, 2018 11:03

faisal00813 added 2 commits May 21, 2018 16:57

Android build scripts for arm arm64 and x86 architectures

a98a882

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com> (cherry picked from commit 2d681f6) Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com> Updated READMEs

Fixed dependency build script

bb4bede

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

faisal00813 force-pushed the android_builds branch from 686e877 to bb4bede Compare May 21, 2018 11:29

Merge branch 'master' into android_builds

1ea665b

faisal00813 and others added 8 commits May 21, 2018 18:22

Fixed external storage path for universal compatibility

6e0d31a

Signed-off-by: Mohammad Abdul Sami faisal00813@gmail.com Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

upgraded log crate and env_logger to work with android_logger

fbd59c0

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Merge remote-tracking branch 'upstream/master' into android_builds

56fe80d

# Conflicts: # libindy/Cargo.lock # libindy/Cargo.toml # libindy/build.rs

Added bundled feature to rusqlite

1062c97

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Merge remote-tracking branch 'origin/android_builds' into android_builds

9a50823

Merge branch 'master' into android_builds

a6e37b8

Fixed a log statement

326ffef

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

vimmerru reviewed Jun 5, 2018

View reviewed changes

faisal00813 added 2 commits June 7, 2018 17:05

- Added build scripts for libnullpay for android support

2c8045c

- Modified to plugged/mod.rs to support arm64 Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Merge branch 'master' into android_builds

2888dd6

vimmerru reviewed Jun 13, 2018

View reviewed changes

- simplified logic to find the android target in build.rs

1a7c0ec

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

vimmerru reviewed Jun 13, 2018

View reviewed changes

faisal00813 added 4 commits June 15, 2018 23:13

Now sqlite is staically linked only on android platform.

c647217

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Now sqlite is staically linked for android and ios.

b8a8d0e

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Updated openssl scripts to checkout the tag from build git repository

33a609d

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Updated version of libsodium in build scripts

0d7dca8

[WIP] updated build scripts to download and use prebuilt dependencies Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

faisal00813 and others added 4 commits June 26, 2018 18:23

Moved the logic of creating the directory in android environment to a…

8279283

…nother function Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Now libindy panics when it cannot find path in EXTERNAL_STORAGE varia…

9a537d1

…ble on android. Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Added prebuild downloads capability in build.withoutdocker.sh script

7d2b666

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Merge branch 'master' into android_builds

4eb089b

jovfer suggested changes Jun 27, 2018

View reviewed changes

faisal00813 added 6 commits June 27, 2018 18:11

Merge branch 'android_builds' of /~https://github.com/faisal00813/indy-sdk

2ec128d

into android_builds

Changed match to if else in build.rs

3ee6359

updated android-build.doc Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

moved android declaration of rusqlite for better visibility

a83969a

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

moved android declaration and general rusqlite togather to bottom of …

adfcde4

…cargo.toml for better visibility Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Reverted: moved android declaration of rusqlite for better visibility

f5f8d69

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

Fixed libnullpay build.rs

7ba938c

Signed-off-by: Mohammad Abdul Sami <faisal00813@gmail.com>

jovfer approved these changes Jun 28, 2018

View reviewed changes

Merge branch 'master' into android_builds

1774122

vimmerru approved these changes Jun 28, 2018

View reviewed changes

vimmerru merged commit 0518ecf into hyperledger-archives:master Jun 28, 2018

Android builds arm arm64 and x86 #777

Android builds arm arm64 and x86 #777

Conversation

faisal00813 commented May 21, 2018 • edited Loading

lodo1995 commented May 21, 2018

faisal00813 commented Jun 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jovfer Jun 27, 2018 • edited Loading

Choose a reason for hiding this comment

AxelNennker commented Jun 13, 2018 • edited Loading

vimmerru commented Jun 15, 2018

vimmerru commented Jun 15, 2018

vimmerru commented Jun 15, 2018

vimmerru commented Jun 15, 2018

AxelNennker commented Jun 22, 2018

faisal00813 commented Jun 26, 2018

faisal00813 commented Jun 26, 2018

AxelNennker commented Jun 26, 2018

faisal00813 commented Jun 26, 2018 • edited Loading

jovfer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jovfer Jun 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jovfer commented Jun 27, 2018

faisal00813 commented May 21, 2018 •

edited

Loading

jovfer Jun 27, 2018 •

edited

Loading

AxelNennker commented Jun 13, 2018 •

edited

Loading

faisal00813 commented Jun 26, 2018 •

edited

Loading

jovfer Jun 27, 2018 •

edited

Loading