Standalone Java bindings #3503

TobiasDeBruijn · 2021-01-18T17:01:17Z

Added standalone Java bindings, that do not rely on Android
Script to produce and collect all required files for usage in the end users own projects
Instructions on how to use and example on how to use

lissyx · 2021-01-18T21:11:19Z

native_client/java/Makefile

@@ -1,31 +1,178 @@
-.PHONY: clean apk-clean
+# CMAKE generated file: DO NOT EDIT!


You cant over write our makefile

You're correct yeah, that one should not be overriden, because it's not used for the standalone java. Will fix.

lissyx · 2021-01-19T08:34:20Z

native_client/java/Makefile

-
-bindings: clean ds-swig
-	$(DS_SWIG_ENV) swig -c++ -java -package org.deepspeech.libdeepspeech -outdir libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/ -o jni/deepspeech_wrap.cpp jni/deepspeech.i
+.PHONY: clean apk-clean


please check your commit, this file seems to be removed / added, it's adding noise to the PR

lissyx · 2021-01-19T08:34:50Z

native_client/java/README.md

+### Notes
+- At this moment, this will only run on x86-64 Linux.
+
+### Example usage


please put this into a file directly

lissyx · 2021-01-19T08:37:25Z

...nt/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/DeepSpeech_Error_Codes.java

@@ -1,6 +1,6 @@
 /* ----------------------------------------------------------------------------
 * This file was automatically generated by SWIG (http://www.swig.org).
- * Version 4.0.1
+ * Version 3.0.12


this shows you have not used the swig version we provide, please do so

lissyx · 2021-01-19T08:38:42Z

...ent/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/SWIGTYPE_p_ModelState.java

@@ -0,0 +1,26 @@
+/* ----------------------------------------------------------------------------


SWIG generated should not be committed

lissyx · 2021-01-19T08:38:51Z

...java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/SWIGTYPE_p_StreamingState.java

@@ -0,0 +1,26 @@
+/* ----------------------------------------------------------------------------


same, no swig generated

lissyx · 2021-01-19T08:38:55Z

...t/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/SWIGTYPE_p_p_ModelState.java

@@ -0,0 +1,26 @@
+/* ----------------------------------------------------------------------------


same, no swig generated

lissyx · 2021-01-19T08:38:59Z

...va/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/SWIGTYPE_p_p_StreamingState.java

@@ -0,0 +1,26 @@
+/* ----------------------------------------------------------------------------


same, no swig generated

lissyx · 2021-01-19T08:39:07Z

native_client/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/TokenMetadata.java

@@ -1,16 +1,13 @@
 /* ----------------------------------------------------------------------------
 * This file was automatically generated by SWIG (http://www.swig.org).
- * Version 4.0.1
+ * Version 3.0.12


wrong version

lissyx · 2021-01-19T08:39:13Z

native_client/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/impl.java

@@ -0,0 +1,157 @@
+/* ----------------------------------------------------------------------------


same, no swig generated

lissyx · 2021-01-19T08:39:26Z

native_client/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/implJNI.java

@@ -0,0 +1,55 @@
+/* ----------------------------------------------------------------------------


same, no swig generated

lissyx · 2021-01-19T08:39:54Z

native_client/java/setup.sh

@@ -0,0 +1,115 @@
+#!/bin/sh


we should not require this kind of script

A makefile would be a better solution probably. But I don't know how to work with a Makefile, and have not figured out yet how to make CMake interact with Gradle without using Android

lissyx · 2021-01-19T10:29:32Z

Let me finish my sport, i might have a better idea than gradle composition

lissyx · 2021-01-20T12:57:34Z

native_client/java/Makefile


 include ../definitions.mk

 ARCHS := $(shell grep 'ABI_FILTERS' libdeepspeech/gradle.properties | cut -d'=' -f2 | sed -e 's/;/ /g')
 GRADLE ?= ./gradlew

-all: apk
+all: apk jre
+android: apk


we should rather just change apk to android instead of adding another layer

lissyx · 2021-01-20T12:57:40Z

native_client/java/Makefile

-all: apk
+all: apk jre
+android: apk
+standalone: jre


lissyx · 2021-01-20T12:58:37Z

native_client/java/Makefile

@@ -20,12 +22,50 @@ libdeepspeech/libs/%/libdeepspeech.so:
 	-mkdir libdeepspeech/libs/$*/
 	cp ${TFDIR}/bazel-out/$*-*/bin/native_client/libdeepspeech.so libdeepspeech/libs/$*/

-apk: apk-clean bindings $(patsubst %,libdeepspeech/libs/%/libdeepspeech.so,$(ARCHS))
+apk-prepare:


this is weird and likely error prone, we should rather have libdeepspeech_android/build.gradle and use that directly instead of doing this cp dance

lissyx · 2021-01-20T12:59:25Z

native_client/java/Makefile

+	$(GRADLE) build
+
+jre: jre-prepare jre-collect jre-restore-makefile jre-clean
+jre-prepare: $(patsubst %,libdeepspeech/libs/%/libdeepspeech.so,$(ARCHS))


lissyx · 2021-01-20T13:00:15Z

native_client/java/Makefile

 	$(GRADLE) build

+jre-collect: jre-gradle


Why this one?

lissyx · 2021-01-20T13:00:39Z

native_client/java/Makefile

+	cp libdeepspeech/libs/x86_64/libdeepspeech.so build/
+	cp libdeepspeech/build/libs/libdeepspeech.jar build/
+
+jre-clean:


cleaning step should be next to apk-clean

lissyx · 2021-01-20T13:02:24Z

native_client/java/Makefile

 maven-bundle: apk
 	$(GRADLE) uploadArchives
 	$(GRADLE) zipMavenArtifacts

 bindings: clean ds-swig
-	$(DS_SWIG_ENV) swig -c++ -java -package org.deepspeech.libdeepspeech -outdir libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/ -o jni/deepspeech_wrap.cpp jni/deepspeech.i
+	$(DS_SWIG_ENV) swig -c++ -java -package org.deepspeech.libdeepspeech -outdir libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/ -o jni/deepspeech_wrap.cpp jni/deepspeech.i


don't change line ending

lissyx · 2021-01-20T13:02:46Z

native_client/java/README.md

+
+``make android``
+
+>Note: The current example app in `./App` is not up to date with the latest changes to the bindings!


what is this ?

lissyx · 2021-01-20T13:03:30Z

native_client/java/gradle/wrapper/gradle-wrapper.properties

@@ -1,5 +1,5 @@
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-6.5.1-bin.zip


can we avoid updating gradle wrapper? this adds noise and is not really needed

lissyx · 2021-01-20T13:03:45Z

native_client/java/gradlew

@@ -1,5 +1,21 @@
 #!/usr/bin/env sh

+#


same, avoid updating gradlew for the moment

lissyx · 2021-01-20T13:03:59Z

native_client/java/gradlew.bat

@@ -1,3 +1,19 @@
+@rem


same, avoid updating for the moment

lissyx · 2021-01-20T13:05:53Z

native_client/java/libdeepspeech/CMakeLists_standalone.txt

+
+add_library( deepspeech-jni SHARED ../jni/deepspeech_wrap.cpp )
+
+find_library(deepspeech-lib NAMES deepspeech PATHS ${CMAKE_SOURCE_DIR}/libs/x86_64/ REQUIRED)


hard coded arch is something we will want to fix later, I think

lissyx · 2021-01-20T13:06:41Z

native_client/java/libdeepspeech/gradle.properties

@@ -1 +1 @@
-ABI_FILTERS = arm64-v8a;armeabi-v7a;x86_64
+ABI_FILTERS = x86_64


Don't change this please (and avoid changing chmod).

lissyx · 2021-01-20T13:08:20Z

...ve_client/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/DeepSpeechModel.java

+    * Create a new DeepSpeechModel object and load the native libraries.<br>
+    * Use on: Android
+    */
+    public DeepSpeechModel() {


This will change the API, what is wrong with static {} ?

Static would always load the libraries that specific way. The way the libraries are loaded for Android don't play nice with standalone.

For end-developers nothing should change though, since the load libraries function is now called in the constructor, for if they dont specify the boolean nothing changes

lissyx · 2021-01-20T13:08:47Z

...ve_client/java/libdeepspeech/src/main/java/org/deepspeech/libdeepspeech/DeepSpeechModel.java

+    * Use on: Android, Standalone
+    * @param loadNativeLibrariesManually if true, the native libraries will not be loaded. The developer must do this manually then.
+    */
+    public DeepSpeechModel(boolean loadNativeLibrariesManually) {


I don't think this is something we want

lissyx · 2021-01-20T13:09:32Z

native_client/java/standalone.md

@@ -0,0 +1,193 @@
+# DeepSpeech for standalone Java


This should not be a .md sitting in the directory but rather a part of doc/

lissyx · 2021-01-20T13:11:26Z

native_client/java/standalone.md

+
+### Adding DeepSpeech to your Java project
+>Note: You can do this on your own way too, but this works with the example usage code provided below.
+1. Copy `libdeepspeech.jar` from `./build/` into `{YOUR PROJECT ROOT}/libs/`


please make sure we don't push people to re-building code themselves except when really required, .jar should have everything bundled as well and people should pull from repos.

native_client/java/standalone.md

lissyx · 2021-01-20T13:13:04Z

native_client/java/standalone.md

+//This class facilitates the loading of the native libraries.
+//The implementation in here works with the Gradle guide provided in the README.md document.
+//However you can modify this to whatever you wish.
+public class LibDeepSpeech extends DeepSpeechModel {


the user should not have to do this, as I explained earlier, it needs to be taken care of in DeepSpeechModel class

Yep correct. it's no longer up-to-date. Since the DeepSpeechModel class is no longer abstract that can go away. Though the library loading is better off in the hands of the user for non-Android, since it differs per project how it's done. Although if we figure out how to ship the .so files in the libdeepspeech jar, then we can 'standardize' it

…etup script

…sible having 'vanilla' Java and the Android bits side by side

- Fixed up Android's Makefile, this got overriden by CMake for the standalone version - Seperate build.gradle and CMakeList.txt for Android and standalone - Changed DeepSpeechModel.java to give control of library loading back to the user - Updated README.md to reflect changes to DeepSpeechModel.java

…on for building for Android

- Split out README.md into standalone.md, for all the bits that are standalone-specific - Removed all automatically generated java files (generated by SWIG) - Updated .gitignore to reflect the change above - Updated DeepSpeechModel.java to be API-compatible with how it was, whilst also allowing the standalone to function (allowing you to load the libraries yourself, if wanted)

- Replaced setup.sh with target `standalone` in Makefile - Updated documentation accordingly - Updated CMakeLists_standalone to work with it now being used from Make (some path things for libdeepspeech.so) - Fixed some wrongly formatted javadoc in DeepSpeechModel.java

- Split out libdeepspeech into _android and _jre, for the respective versions - Moved settings.gradle to both versions, and modified them appropriately - Removed .so files, which were accidentally included in the previous commit, same goes for Makefile.original - Updated .gitignore's to work with the new structure - libdeepspeech_android is exactly as it is currently in mozilla:master - Updated Makefile to work with the new structure

lissyx · 2021-02-17T13:05:15Z

native_client/java/build.gradle.standalone

+
+    repositories {
+        google()
+        jcenter()


jcenter() is dying :'(

Not sure what we can do about that. Will need to check out which dependencies are from jcenter, I dont think we have any since the Android stuff is pulled from Google

Liuhaai · 2021-03-25T00:11:52Z

Hi TheDutchMC,
I face an issue after make standalone. The file tensorflow/bazel-out/x86_64-*/bin/native_client/libdeepspeech.so is not found.

TobiasDeBruijn · 2021-03-25T08:11:53Z

Hi TheDutchMC,
I face an issue after make standalone. The file tensorflow/bazel-out/x86_64-*/bin/native_client/libdeepspeech.so is not found.

Hey there, have you built DeepSpeech before trying to create the java bindings? See the documentation here

Liuhaai · 2021-03-25T21:18:16Z

Hi TheDutchMC,
I face an issue after make standalone. The file tensorflow/bazel-out/x86_64-*/bin/native_client/libdeepspeech.so is not found.

Hey there, have you built DeepSpeech before trying to create the java bindings? See the documentation here

Except for compiling DeepSpeech, which platform should be built? If you could upload libdeepspeech.jar and *.so files, it would be convenient to test the Java bindings.

TobiasDeBruijn · 2021-03-25T21:54:49Z

For now only x86-64 (linux I believe) works, though you could easily modify the Makefile for your platform. This is a WIP.

Keep in mind the code for the standalone jdk is probably going to change a bit, I dont like the way library loading is done at the moment.

I should have some time to look at it again tomorrow afternoon and Saturday.

Liuhaai · 2021-03-25T21:57:16Z

For now only x86-64 (linux I believe) works, though you could easily modify the Makefile for your platform. This is a WIP.

Keep in mind the code for the standalone jdk is probably going to change a bit, I dont like the way library loading is done at the moment.

I should have some time to look at it again tomorrow afternoon and Saturday.

Thanks for your work!

lissyx reviewed Jan 18, 2021

View reviewed changes

lissyx reviewed Jan 19, 2021

View reviewed changes