sqlite/ext/jni

SQLite3 via JNI

This directory houses a Java Native Interface (JNI) binding for the sqlite3 API. If you are reading this from the distribution ZIP file, links to resources in the canonical source tree will note work. The canonical copy of this file can be browsed at:

https://sqlite.org/src/doc/trunk/ext/jni/README.md

Technical support is available in the forum:

https://sqlite.org/forum

FOREWARNING: this subproject is very much in development and subject to any number of changes. Please do not rely on any information about its API until this disclaimer is removed. The JNI bindgins released with version 3.43 are a "tech preview" and 3.44 will be "final," at which point strong backward compatibility guarantees will apply.

Project goals/requirements:

• A 1-to-1(-ish) mapping of the C API to Java via JNI, insofar as cross-language semantics allow for. A closely-related goal is that the C documentation should be usable as-is, insofar as possible, for the JNI binding.

• Support Java as far back as version 8 (2014).

• Environment-independent. Should work everywhere both Java and SQLite3 do.

• No 3rd-party dependencies beyond the JDK. That includes no build-level dependencies for specific IDEs and toolchains. We welcome the addition of build files for arbitrary environments insofar as they neither interfere with each other nor become a maintenance burden for the sqlite developers.

Non-goals:

• Creation of high-level OO wrapper APIs. Clients are free to create them off of the C-style API.

Hello World

import org.sqlite.jni.*;
import static org.sqlite.jni.SQLite3Jni;
...
OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
int rc = sqlite3_open(":memory:", out);
final sqlite3 db = out.take();
if( 0 != rc ){
  if( null != db ){
    System.out.print("Error opening db: "+sqlite3_errmsg(db));
    sqlite3_close(db);
  }else{
    System.out.print("Error opening db: rc="+rc);
  }
  ... handle error ...
}

... use db ...

sqlite3_close_v2(db);

Building

The canonical builds assumes a Linux-like environment and requires:

• GNU Make
• A JDK supporting Java 8 or higher
• A modern C compiler. gcc and clang should both work.

Put simply:

$ export JAVA_HOME=/path/to/jdk/root
$ make
$ make test
$ make clean

The jar distribution can be created with make jar.

One-to-One(-ish) Mapping to C

This JNI binding aims to provide as close to a 1-to-1 experience with the C API as cross-language semantics allow. Changes are necessarily made where cross-language semantics do not allow a 1-to-1, and judiciously made where a 1-to-1 mapping would be unduly cumbersome to use in Java.

Golden Rule: Never Throw from Callbacks (Unless...)

Client-defined callbacks must never throw exceptions unless very explicitly documented as being throw-safe. Exceptions are generally reserved for higher-level bindings which are constructed to specifically deal with them and ensure that they do not leak C-level resources.

Unwieldy Constructs are Re-mapped

Some constructs, when modelled 1-to-1 from C to Java, are unduly clumsy to work with in Java because they try to shoehorn C's way of doing certain things into Java's wildly different ways. The following subsections cover those, starting with a verbose explanation and demonstration of where such changes are "really necessary"...

Custom Collations

A prime example of where interface changes for Java are necessary for usability is registration of a custom collation:

// C:
int sqlite3_create_collation(sqlite3 * db, const char * name, int eTextRep,
                             void *pUserData,
                             int (*xCompare)(void*,int,void const *,int,void const *));

int sqlite3_create_collation_v2(sqlite3 * db, const char * name, int eTextRep,
                                void *pUserData,
                                int (*xCompare)(void*,int,void const *,int,void const *),
                                void (*xDestroy)(void*));

The pUserData object is optional client-defined state for the xCompare() and/or xDestroy() callback functions, both of which are passed that object as their first argument. That data is passed around "externally" in C because that's how C models the world. If we were to bind that part as-is to Java, the result would be awkward to use (^Yes, we tried this.):

// Java:
int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
                             Object pUserData, xCompareType xCompare);

int sqlite3_create_collation_v2(sqlite3 db, String name, int eTextRep,
                                Object pUserData,
                                xCompareType xCompare, xDestroyType xDestroy);

The awkwardness comes from (A) having two distinctly different objects for callbacks and (B) having their internal state provided separately, which is ill-fitting in Java. For the sake of usability, C APIs which follow that pattern use a slightly different Java interface:

int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
                             Collation collation);

Where the Collation class has an abstract xCompare() method and no-op xDestroy() method which can be overridden if needed, leading to a much more Java-esque usage:

int rc = sqlite3_create_collation(db, "mycollation", SQLITE_UTF8, new Collation(){

  // Required comparison function:
  @Override public int xCompare(byte[] lhs, byte[] rhs){ ... }

  // Optional finalizer function:
  @Override public void xDestroy(){ ... }

  // Optional local state:
  private String localState1 =
    "This is local state. There are many like it, but this one is mine.";
  private MyStateType localState2 = new MyStateType();
  ...
});

Noting that:

• It is possible to bind in call-scope-local state via closures, if desired, as opposed to packing it into the Collation object.

• No capabilities of the C API are lost or unduly obscured via the above API reshaping, so power users need not make any compromises.

• In the specific example above, sqlite3_create_collation_v2() becomes superfluous because the provided interface effectively provides both the v1 and v2 interfaces, the difference being that overriding the xDestroy() method effectively gives it v2 semantics.

User-defined SQL Functions (a.k.a. UDFs)

The sqlite3_create_function() family of APIs make heavy use of function pointers to provide client-defined callbacks, necessitating interface changes in the JNI binding. The Java API has only one core function-registration function:

int sqlite3_create_function(sqlite3 db, String funcName, int nArgs,
                            int encoding, SQLFunction func);

Design question: does the encoding argument serve any purpose in Java? That's as-yet undetermined. If not, it will be removed.

SQLFunction is not used directly, but is instead instantiated via one of its three subclasses:

• SQLFunction.Scalar implements simple scalar functions using but a single callback.
• SQLFunction.Aggregate implements aggregate functions using two callbacks.
• SQLFunction.Window implements window functions using four callbacks.

Search Tester1.java for SQLFunction for how it's used.

Reminder: see the disclaimer at the top of this document regarding the in-flux nature of this API.

And so on...

Various APIs which accept callbacks, e.g. sqlite3_trace_v2() and sqlite3_update_hook(), use interfaces similar to those shown above.