ANE format for Android - builds and dependencies

[ajwfrost]

Capturing some thoughts here that came out of a fix we just did for ANEs on Android...

Currently on Android:

• most ANEs are Java-based, so you have to build a library, pull it apart for the .jar file, and this then gets duplicated across all of the Android platforms (armv7, armv8, x86, x64) in the ANE.
• currently if you want to include a third party library you add this also as a .jar file in each platform folder
• resources are then included by copying them into a subfolder and adding a notification about the package these should be compiled into

Since we're moving over to an approach that generates an Android Studio style project before building this with the standard Gradle toolchain, it gives us an opportunity to change what we're doing...

Thoughts to be added below please! From an initial discussion with Distriqt there are a couple of ideas:

1. whatever we do should be done in a gradual/backwards-compatible method, so that you don't have to change everything all at once, old ANEs will still work, etc
2. we could add a dependency now in the same way that they're added in Android Studio i.e. a gradle dependency (see below for an example)
3. we could have platform-agnostic ANEs for Android or just change how the NDK/C++ style ones are created without the need for platform-specific folders (I'm not sure there's much excuse for not having .so files for each of the four platforms now.. but in case there are old ANEs that don't have some of them, we should be able to handle that relatively gracefully at runtime..)
4. rather than jar files and resource folders, could we switch to supporting .aar files?

The suggestion for a new set of dependencies for Android libraries would be something like:

    <gradleDependencies>
        <gradleDependency>implementation "com.google.android.gms:play-services-base:17.5.0"</gradleDependency>
    </gradleDependencies> 

Any other thoughts/suggestions, please add them below!

thanks

Andrew

[marchbold]

Android platforms

I think I raised a similar thing with Adobe years ago when we started supporting Intel processors on Android. The duplication seems completely unnecessary.

Even when we do use native code in our extensions we generally put it in a libs folder which then is divided into each of the processor implementations:

Eg from our scanner extension:

|____distriqt.extension.scanner.android.jar
|____distriqt-extension-scanner-res
| |____layout
| | |____scanner_view.xml
| |____values
| | |____distriqt_scanner_strings.xml
|____libs
| |____armeabi-v7a
| | |____libzbarjni.so
| | |____libiconv.so
| |____x86
| | |____libzbarjni.so
| | |____libiconv.so
| |____arm64-v8a
| | |____libzbarjni.so
| | |____libiconv.so
| |____armeabi
| | |____libzbarjni.so
| | |____libiconv.so
| |____x86_64
| | |____libzbarjni.so
| | |____libiconv.so
|____zbar.jar
|____library.swf

And this is then duplicated across each platform inside the ANE. This hasn't been much of a concern though as it's just a filesize issue with the ANE and doesn't affect the output app so we haven't pushed too hard on it.

This definitely could be an improvement in the future, something as simple as adding an Android platform name that is used in all cases that aren't otherwise specified?

<extension xmlns="http://ns.adobe.com/air/extension/24.0">
  <id>com.distriqt.Scanner</id>
  <versionNumber>5.0.58</versionNumber>
  <platforms>
  
    <platform name="Android">
      <applicationDeployment>
        <nativeLibrary>distriqt.extension.scanner.android.jar</nativeLibrary>
        <initializer>com.distriqt.extension.scanner.ScannerExtension</initializer>
        <finalizer>com.distriqt.extension.scanner.ScannerExtension</finalizer>
      </applicationDeployment>
    </platform>

    <!-- other platforms -->

  </platforms>
</extension>

One concern I have here is how older IDE's (I'm thinking FlashBuilder mainly here) will handle this change as they seem to inspect the ANEs for supported platforms and show errors if not available. So I would be cautious jumping into these changes without thorough IDE tests.

Gradle

I think you have summarised the points I suggested fairly well here. What I'd like to be able to do is move our dependency ANEs to a simple gradle line. Eventually this will remove the need for these dependency ANEs altogether. As a migration step we would just remove all packaged dependency libraries and resources in those ANEs and replace with the gradle line. So if I look at the com.google.android.material ANE we would look to change:

<platform xmlns="http://ns.adobe.com/air/extension/24.0">
  <copyright>2019 // distriqt</copyright>
  <packagedDependencies>
    <packagedDependency>material-1.0.0.jar</packagedDependency>
  </packagedDependencies>
  <packagedResources>
    <packagedResource>
      <packageName>com.google.android.material</packageName>
      <folderName>material-1.0.0-res</folderName>
    </packagedResource>
  </packagedResources>
</platform>

to a gradle based one like:

<platform xmlns="http://ns.adobe.com/air/extension/24.0">
  <copyright>2021 // distriqt</copyright>
  <packagedDependencies>
  </packagedDependencies>
  <packagedResources>
  </packagedResources>
  <gradle>
    <repositories>
      <repository>google()</repository>
    </repositories>
    <dependencies>
      <dependency>implementation 'com.google.android.material:material:1.0.0'</dependency>
    </dependencies>
  </gradle>
</platform>

These gradle/dependencies/dependency lines would be added to the app module build.gradle .

Then there is also the requirement of adding the repository that the dependency can be found in. This is added to the project build.gradle in the repositories section, for the above case the material library is in the google() repository.

There are further additions I can think of (around plugins eg for Firebase), but they can perhaps be a phase 2?

It is important that this works with both apk and aab packaging, in order to continue supporting other stores, not just the Google Play Store. So firstly we need to make sure you have the ability to provide a gradle project solution for both packaging outputs. Additionally I can imagine there'd need to be some level of conflict handling if two ANEs specified different versions of a gradle dependency, haven't looked too hard into that detail yet though.

AAR

Using aar as the native library and packaged dependencies would be great! This would add several additional features in combination with a gradle project, such as:

• automatic manifest merging
• resource packaging
• additional asset packaging

However this adds the additional requirement of being able to pass in configuration items. Many libraries require additions to the manifest that are specified in the build.gradle. Currently these are manually added to the application descriptor (and will be part of apm) but if you are going to go down the aar path then this would quickly become a requirement as manifest merging will fail without it.

I would see this as requiring some additions to the android node in the application descriptor.

Priorities

The current ANE structure is "working" so I'd like to make sure that these improvements don't get in the way of producing a stable AIR output, which I think should be number 1 priority. I don't think anything here is critical or blocking development with AIR currently.

Regarding these features however, I would prioritise the work as follows:

1. producing a stable gradle based packaging solution for all Android outputs (apk/ aab)

2. adding 'gradle` options to the android platform xml

3. adding the single Android platform

4. ability to use aar and provide any related configuration through the application descriptor

[uerceg]

Thank you for the hint, I'll definitely give it a shot. And regarding the legacy build method you have mentioned - what's the legacy build method? :)

[marchbold]

When the AIR SDK can't find an android sdk installation it uses the older method of packaging the java classes / resources etc into an apk. This approach doesn't support gradle nor a range of other features. Harman introduced the more modern approach of using gradle and the android sdk directly which is so much better for development.

You can force the legacy mode by a flag in the application descriptor so check that you haven't put that in there and ensure you have supplied a path to the android sdk either via the AIR SDK Manager or by editing the adt.cfg directly.

[marchbold]

Note: You can spot the legacy mode usage by watching the adt debug log.

[uerceg]

Thank you so much for your advices, I will try them as soon as I can and update in here what is that I've managed to do (or not). Seems like we are on the opposite sides of the world, so there's a bit of lag in between. Thank you very much one more time for all the help so far. 🙏

[marchbold]

Happy to help from Australia! :)

[llyRayYoung]

Is this feature implemented now?

[ajwfrost]

I think there are multiple steps here.. I'd say that the first one of these is done (per the above priority list) - but we've not got further with the other ones. We should be able to move forwards to the idea of Gradle options in the Android platform XML files (but this means that such ANEs wouldn't be compatible with the non-Gradle-based packaging of APKs, which some people still use).

Do you have a particular interest in one of the proposed changes?

thanks

[llyRayYoung]

Thanks. I want use gradle like this " implementation 'com.xxx'" in my ane, but i don't know how to implement it

[ajwfrost]

We can't generate implementation 'com.xxx' yet but if you have a dependent library (jar file) in the platform options then we add this via implementation files('libs/com.xxx.jar'). We'll look at getting this next step into a forthcoming release of the AIR SDK though..

[Oldes]

PING @ajwfrost PING @marchbold PING

Was this somehow updated (improved) since recent years? I'm right now making a new ANE for Android and I wonder if I still need to duplicate jar and other resources files for each Android platform (Android-ARM, Android-ARM64, Android-x86, Android-x64) when evaluating adt.jar -package -target ane .... command to pack the ANE.

And as my new ANE needs shared libraries as well, is it now possible to provide just a single libs folder location for all Android system versions (with the standard folder names armeabi-v7a, arm64-v8a, etc) or I must copy the right so version files into each platform folder?

[marchbold]

Must say we haven't tried this yet either. All of our extensions are still setup with all the different variants libs folder distributed accordingly. Will add it into my list to have a look over.

[bobaoapae]

Using aar format will be better if the dependencies could be integrated automatically without need of adding to platform file.

[ajwfrost]

@bobaoapae from my understanding, any AAR dependencies should be listed in its manifest file? I don't think we have this set up, but have just taken a look and we should be able to pick these up. I couldn't actually see a way in Android Studio to have them configured automatically though - i.e. it would require the ANE developer to manually edit the manifest file and add uses-library directives, rather than being anything we could pick up automatically based on the project dependencies in build.gradle..?

[bobaoapae]

It's possible to pick from the build.gradle, check here

If build.gradle of the aar use dependencies as API they will be needed by the final application.

One thing to take care it's of the new format of not list the full dependencie on the gradle file, but in file (libs.versions.toml).

You can check my ane here to see this in action libs.versions.toml and build.gradle.kts.

Android studio don't pick those dependencies automatically because this concept of transitive dependencies it's on the maven side not in gradle process. On maven dependencies that has transitives dependencies they have those declared on they pom.xml file.

If harman will add a way of simplify this process, taking dependencies marked as api from the aar gradle and adding those to the final gradle of the application as implementation will make things more easy, as managing dependencies on two files can cause problems due versions mismatch if don't pay atention when updating.

[marchbold]

This is what you can do in your platform xml, you add in the gradle lines you require. There is no method for this to be included in an aar.

Our build process has a simple script that reads our gradle files and constructs the platform xml from them.

<platform xmlns="http://ns.adobe.com/air/extension/24.0">
  <copyright>2021 // distriqt</copyright>
  <packagedDependencies>
  </packagedDependencies>
  <packagedResources>
  </packagedResources>
  <gradle>
    <repositories>
      <repository>google()</repository>
    </repositories>
    <dependencies>
      <dependency>implementation 'com.google.android.material:material:1.0.0'</dependency>
    </dependencies>
  </gradle>
</platform>