third_party/jni_zero/README.md

# JNI Zero
A zero-overhead (or better!) middleware for JNI.

## Overview
JNI (Java Native Interface) is the mechanism that enables Java code to call
native functions, and native code to call Java functions.

 * Native code calls into Java using apis from `<jni.h>`, which basically mirror
   Java's reflection APIs.
 * Java code calls native functions by declaring body-less functions with the
  `native` keyword, and then calling them as normal Java functions.

JNI Zero generates boiler-plate code with the goal of making our code:
 1. easier to write,
 2. typesafe.
 3. more optimizable.

JNI Zero uses regular expressions to parse .Java files, so don't do
anything too fancy. E.g.:
 * Classes must be either explicitly imported, or are assumed to be in
the same package. To use `java.lang` classes, add an explicit import.
 * Inner classes need to be referenced through the outer class. E.g.:
   `void call(Outer.Inner inner)`

### Exposing Native Methods

There are two ways to have native methods be found by Java:
1) Explicitly register the name -> function pointer mapping using JNI's
   `RegisterNatives()` function.
2) Export the symbols from the shared library, and let the runtime resolve them
   on-demand (using `dlsym()`) the first time a native method is called.

2) Is generally preferred due to a smaller code size and less up-front work, but
1) is sometimes required (e.g. when OS bugs prevent `dlsym()` from working).
Both ways are supported by this tool.

### Exposing Java Methods

Java methods just need to be annotated with `@CalledByNative`. By default the
generated method stubs on the native side are not namespaced. The generated
functions can be put into a namespace using `@JNINamespace("your_namespace")`.

## Usage

### Writing Build Rules
1. Find or add a `generate_jni` target with your .java file, then add this
   `generate_jni` target to your `srcjar_deps` of your `android_library` target:

   ```python
   generate_jni("abcd_jni") {
     sources = [ "path/to/java/sources/with/jni/Annotations.java" ]
   }

   android_library("abcd_java") {
     ...
     # Allows the java files to see the generated `${OriginalClassName}Jni`
     # classes.
     srcjar_deps = [ ":abcd_jni" ]
   }

   source_set("abcd") {
    ...
    # Allows the cpp files to include the generated `${OriginalClassName}_jni.h`
    # headers.
    deps = [ ":abcd_jni" ]
   }
   ```

### Calling Java -> Native

- For each JNI method:
  - C++ stubs are generated that forward to C++ functions that you must write.
    By default the c++ functions you are expected to implement are not
    associated with a class.
  - If the first parameter is a C++ object (e.g.
    `long native${OriginalClassName}`), then the bindings will not call a static
    function but instead cast the variable into a cpp `${OriginalClassName}`
    pointer type and then call a member method with that name on said object.

To add JNI to a class:

1. Create a nested-interface annotated with `@NativeMethods` that contains
   the declaration of the corresponding static methods you wish to have
   implemented.
2. Call native functions using `${OriginalClassName}Jni.get().${method}`
3. In C++ code, #include the header `${OriginalClassName}_jni.h`. (The path will
   depend on the location of the `generate_jni` BUILD rule that lists your Java
   source code.) Only include this header from a single `.cc` file as the
   header defines functions. That `.cc` must implement your native code by
   defining non-member functions named `JNI_${OriginalClassName}_${UpperCamelCaseMethod}`
   for static methods and member functions named `${OriginalClassName}::${UpperCamelCaseMethod}`
   for non-static methods. Member functions need be declared in the header
   file as well.

Example:
#### Java
```java
class MyClass {
  // Cannot be private. Must be package or public.
  @NativeMethods
  /* package */ interface Natives {
    void foo();
    double bar(int a, int b);
    // Either the |MyClass| part of the |nativeMyClass| parameter name must
    // match the native class name exactly, or the method annotation
    // @NativeClassQualifiedName("MyClass") must be used.
    //
    // If the native class is nested, use
    // @NativeClassQualifiedName("FooClassName::BarClassName") and call the
    // parameter |nativePointer|.
    void nonStatic(long nativeMyClass);
  }

  void callNatives() {
    // MyClassJni is generated by the generate_jni rule.
    // Storing MyClassJni.get() in a field defeats some of the desired R8
    // optimizations, but local variables are fine.
    Natives jni = MyClassJni.get();
    jni.foo();
    jni.bar(1,2);
    jni.nonStatic(mNativePointer);
  }
}
```
#### C++
```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

class MyClass {
public:
  void NonStatic(JNIEnv* env);
}

// Notice that unlike Java, function names are capitalized in C++.
// Static function names should follow this format and don't need to be declared.
void JNI_MyClass_Foo(JNIEnv* env) { ... }
void JNI_MyClass_Bar(JNIEnv* env, jint a, jint b) { ... }

// Member functions need to be declared.
void MyClass::NonStatic(JNIEnv* env) { ... }
```

### Calling Native -> Java

Because the generated header files contain definitions as well as declarations,
the must not be `#included` by multiple sources. If there are Java functions
that need to be called by multiple sources, one source should be chosen to
expose the functions to the others via additional wrapper functions.

1. Annotate some methods with `@CalledByNative`, the generator will now generate
   stubs in `${OriginalClassName}_jni.h` header to call into those java methods
   from cpp.
   * Inner class methods must provide the inner class name explicitly
     (ex. `@CalledByNative("InnerClassName")`)

2. In C++ code, `#include` the header `${OriginalClassName}_jni.h`. (The path
   will depend on the location of the `generate_jni` build rule that lists your
   Java source code). That `.cc` can call the stubs with their generated name
   `JAVA_${OriginalClassName}_${UpperCamelCaseMethod}`.

Note: For test-only methods, use `@CalledByNativeForTesting` which will ensure
that it is stripped in our release binaries.

### Automatic Type Conversions using @JniType

Normally, Java types map to C++ types from `<jni.h>` (e.g. `jstring` for
`java.lang.String`). The first thing most people do is convert the jni spec
types into standard C++ types.

`@JniType` to the rescue. By annotating a parameter or a return type with
`@JniType("cpp_type_here")` the generated code will automatically convert from
the jni type to the type listed inside the annotation. See example:

#### Original Code:
```java
class MyClass {
  @NativeMethods
  interface Natives {
    void foo(
            String string,
            String[] strings,
            MyClass obj,
            MyClass[] objs)
  }
}
```

```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

void JNI_MyClass_Foo(JNIEnv* env, const JavaParamRef<jstring>&, const JavaParamRef<jobjectArray>&, const JavaParamRef<jobject>&, JavaParamRef<jobjectArray>&) {...}
```

#### After using `@JniType`
```java
class MyClass {
  @NativeMethods
  interface Natives {
    void foo(
            @JniType("std::string") String convertedString,
            @JniType("std::vector<std::string>") String[] convertedStrings,
            @JniType("myModule::CPPClass") MyClass convertedObj,
            @JniType("std::vector<myModule::CPPClass>") MyClass[] convertedObjects);
  }
}
```
```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

void JNI_MyClass_Foo(JNIEnv* env, std::string&, std::vector<std::string>>&, myModule::CPPClass&, std::vector<myModule::CPPClass>&) {...}
```

#### Implementing Conversion Functions

Conversion functions must be defined for all types that appear in `@JniType`.
Forgetting to add one will result in errors at link time.

```c++
// The conversion function primary templates.
template <typename O>
O FromJniType(JNIEnv*, const JavaRef<jobject>&);
template <typename O>
O FromJniType(JNIEnv*, const JavaRef<jstring>&);
template <typename O>
ScopedJavaLocalRef<jobject> ToJniType(JNIEnv*, const O&);
```

An example conversion function can look like:
```c++
#include "third_party/jni_zero/jni_zero.h"

namespace jni_zero {
template <>
EXPORT std::string FromJniType<std::string>(
    JNIEnv* env,
    const JavaRef<jstring>& input) {
  // Do the actual conversion to std::string.
}

template <>
EXPORT ScopedJavaLocalRef<jstring> ToJniType<std::string>(
    JNIEnv* env,
    const std::string& input) {
  // Do the actual conversion from std::string.
}
}  // namespace jni_zero
```

If a conversion function is missing, you will get a linker error since we
forward declare the conversion functions before using them.

#### Array Conversion Functions

Array conversion functions look different due to the partial specializations.
The `ToJniType` direction also takes a `jclass` parameter which is the class of the
array elements, because java requires it when creating a non-primitive array.

```c++
template <typename O>
struct ConvertArray {
  static O FromJniType(JNIEnv*, const JavaRef<jobjectArray>&);
  static ScopedJavaLocalRef<jobjectArray> ToJniType(JNIEnv*, const O&, jclass);
};
```

JniZero provides implementations for partial specializations to wrap and unwrap
`std::vector` for object arrays and some primitive arrays.

#### Nullability

All non-primitive default JNI C++ types (e.g. `jstring`, `jobject`) are pointer
types (i.e. nullable). Some C++ types (e.g. `std::string`) are not pointer types
and thus cannot be `nullptr`. This means some conversion functions that return
non-nullable types have to handle the situation where the passed in java type is
null.

You can get around this by having the conversion be to `std::optional<T>` rather
than just `T` if `T` is not a nullable type.


### Testing Mockable Natives

1. Add the `JniMocker` rule to your test.
2. Call `JniMocker#mock` in a `setUp()` method for each interface you want to
   stub out.

`JniMocker` will reset the stubs during `tearDown()`.

```java
/**
 * Tests for {@link AnimationFrameTimeHistogram}
 */
@RunWith(BaseRobolectricTestRunner.class)
@Config(manifest = Config.NONE)
public class AnimationFrameTimeHistogramTest {
    @Rule
    public JniMocker mocker = new JniMocker();

    @Mock
    AnimationFrameTimeHistogram.Natives mNativeMock;

    @Before
    public void setUp() {
        MockitoAnnotations.initMocks(this);
        mocker.mock(AnimationFrameTimeHistogramJni.TEST_HOOKS, mNativeMock);
    }

    @Test
    public void testNatives() {
        AnimationFrameTimeHistogram hist = new AnimationFrameTimeHistogram("histName");
        hist.startRecording();
        hist.endRecording();
        verify(mNativeMock).saveHistogram(eq("histName"), any(long[].class), anyInt());
    }
}
```

If a native method is called without setting a mock in a unit test, an
`UnsupportedOperationException` will be thrown.

### Special case: DFMs
DFMs have their own generated `GEN_JNI`s, which are `<module_name>_GEN_JNI`. In
order to get your DFM's JNI to use the `<module_name>` prefix, you must add your
module name into the argument of the `@NativeMethods` annotation.

So, for example, say your module was named `test_module`. You would annotate
your `Natives` interface with `@NativeMethods("test_module")`, and this would
result in `test_module_GEN_JNI`.


### Testing for readiness: use `get()`

JNI Generator automatically produces asserts that verify that the Natives interface can be safely
called. These checks are compiled out of Release builds, making these an excellent way to determine
whether your code is called safely.

It is not sufficient, however, to use `<Class>Jni.get()` to guarantee native is initialized - it is
only a debugging tool to ensure that you're using native after native is loaded.

If you expect your code to be called by an external caller, it's often helpful to know _ahead of
time_ that the context is valid (ie. either native libraries are loaded or mocks are installed).
In this case it is helpful to call `get()` method, that performs all the Debug checks listed
above, but does not instantiate a new object for interfacing Native libraries.
Note that the unused value returned by the `get()` method will be optimized away in release builds
so there's no harm in ignoring it.

#### Addressing `Jni.get()` exceptions.

When you identify a scenario leading to an exception, relocate (or defer) the appropriate call to
be made to a place where (or time when) you know the native libraries have been initialized (eg.
`onStartWithNative`, `onNativeInitialized` etc).

Please avoid calling `LibraryLoader.isInitialized()` / `LibraryLoader.isLoaded()` in new code.
Using `LibraryLoader` calls makes unit-testing more difficult:
* this call can not verify whether Mock object is used, making the use of mocks more complicated,
* using `LibraryLoader.setLibrariesLoadedForNativeTests()` alters the state for subsequently
executed tests, inaccurately reporting flakiness and failures of these victim tests.
* Introducing `LibraryLoader.is*()` calls in your code immediately affects all callers, forcing
the authors of the code up the call stack to override `LibraryLoader` internal state in order to be
able to unit-test their code.

However, if your code is going to be called both before and after native is initialized, you are
forced to call `LibraryLoader.isInitialized()` to be able to differentiate. Calling
`<Class>Jni.get()` only provides assertions, and will fail in debug builds if you call it when
native isn't ready.

### Java Objects and Garbage Collection

All pointers to Java objects must be registered with JNI in order to prevent
garbage collection from invalidating them.

For Strings & Arrays - it's common practice to use the `//base/android/jni_*`
helpers to convert them to `std::vectors` and `std::strings` as soon as
possible.

For other objects - use smart pointers to store them:
 * `ScopedJavaLocalRef<>` - When lifetime is the current function's scope.
 * `ScopedJavaGlobalRef<>` - When lifetime is longer than the current function's
   scope.
 * `JavaObjectWeakGlobalRef<>` - Weak reference (do not prevent garbage
   collection).
 * `JavaParamRef<>` - Use to accept any of the above as a parameter to a
   function without creating a redundant registration.

### Additional Guidelines / Advice

Minimize the surface API between the two sides. Rather than calling multiple
functions across boundaries, call only one (and then on the other side, call as
many little functions as required).

If a Java object "owns" a native one, store the pointer via
`"long mNativeClassName"`. Ensure to eventually call a native method to delete
the object. For example, have a `close()` that deletes the native object.

The best way to pass "compound" types across in either direction is to
create an inner class with PODs and a factory function. If possible, mark
all the fields as "final".

## Build Rules

 * `generate_jni` - Generates a header file with stubs for given `.java` files
 * `generate_jar_jni` - Generates a header file with stubs for a given `.jar`
   file
 * `generate_jni_registration` - Generates a header file with functions to
   register native-side JNI methods.

Refer to [//build/config/android/rules.gni](https://cs.chromium.org/chromium/src/build/config/android/rules.gni)
for more about the GN templates.

## Changing JNI Zero

 * Python tests live in `test/integration_tests.py`
 * A working demo app exists as `//third_party/jni_zero/sample:jni_zero_sample_apk`