sqlite/www/34to35.tcl

#
# Run this TCL script to generate HTML for the goals.html file.
#
set rcsid {$Id: 34to35.tcl,v 1.4 2007/10/01 13:54:11 drh Exp $}
source common.tcl
header {SQLite Changes From Version 3.4.2 To 3.5.0}

proc CODE {text} {
  puts "<blockquote><pre>"
  puts $text
  puts "</pre></blockquote>"
}
proc SYNTAX {text} {
  puts "<blockquote><pre>"
  set t2 [string map {& &amp; < &lt; > &gt;} $text]
  regsub -all "/(\[^\n/\]+)/" $t2 {</b><i>\1</i><b>} t3
  puts "<b>$t3</b>"
  puts "</pre></blockquote>"
}
proc IMAGE {name {caption {}}} {
  puts "<center><img src=\"$name\">"
  if {$caption!=""} {
    puts "<br>$caption"
  }
  puts "</center>"
}
proc PARAGRAPH {text} {
  # regsub -all "/(\[a-zA-Z0-9\]+)/" $text {<i>\1</i>} t2
  #regsub -all "\\*(\[^\n*\]+)\\*" $text {<tt><b><big>\1</big></b></tt>} t3
  regsub -all {\[([^]\n]+)\]} $text {[resolve_link \1]} t3
  puts "<p>[subst -novar -noback $t3]</p>\n"
}
proc resolve_link {args} {
  set a2 [split $args |]
  set id [string trim [lindex $a2 0]]
  if {[lindex $a2 1]==""} {
    set display [string trim [lindex $a2 0]]
  } else {
    set display [string trim [lrange $a2 1 end]]
  }
  regsub -all {[^a-zA-Z0-9_]} $id {} id
  return "<a href=\"capi3ref.html#$id\">$display</a>"
}
set level(0) 0
set level(1) 0
proc HEADING {n name {tag {}}} {
  if {$tag!=""} {
    puts "<a name=\"$tag\">"
  }
  global level
  incr level($n)
  for {set i [expr {$n+1}]} {$i<10} {incr i} {
    set level($i) 0
  }
  if {$n==0} {
    set num {}
  } elseif {$n==1} {
    set num $level(1).0
  } else {
    set num $level(1)
    for {set i 2} {$i<=$n} {incr i} {
      append num .$level($i)
    }
  }
  incr n 1
  puts "<h$n>$num $name</h$n>"
}

HEADING 0 {Moving From SQLite 3.4.2 to 3.5.0}

PARAGRAPH {
  SQLite version 3.5.0 introduces a new OS interface layer that
  is incompatible with all prior versions of SQLite.  In addition,
  a few existing interfaces have been generalized to work across all
  database connections within a process rather than just all
  connections within a thread.  The purpose of this article
  is to describe the changes to 3.5.0 in detail so that users
  of prior versions of SQLite can judge what, if any, effort will
  be required to upgrade to newer versions.
}

HEADING 1 {Overview Of Changes}

PARAGRAPH {
  A quick enumeration of the changes in SQLite version 3.5.0
  is provide here.  Subsequent sections will describe these
  changes in more detail.
}
PARAGRAPH {
  <ol>
  <li>The OS interface layer has been completely reworked:
  <ol type="a">
  <li>The undocumented <b>sqlite3_os_switch()</b> interface has
      been removed.</li>
  <li>The <b>SQLITE_ENABLE_REDEF_IO</b> compile-time flag no longer functions.
      I/O procedures are now always redefinable.</li>
  <li>Three new objects are defined for specifying I/O procedures:
      [sqlite3_vfs], [sqlite3_file], and [sqlite3_io_methods].</li>
  <li>Three new interfaces are used to create alternative OS interfaces:
      [sqlite3_vfs_register()], [sqlite3_vfs_unregister()], and
      [sqlite3_vfs_find()].</li>
  <li>A new interface has been added to provided additional control over
      the creation of new database connections: [sqlite3_open_v2()].
      The legacy interfaces of [sqlite3_open()] and
      [sqlite3_open16()] continue to be fully supported.</li>
  </ol></li>
  <li>The optional shared cache and memory management features that
      were introduced in version 3.3.0 can now be used across multiple
      threads within the same process.  Formerly, these extensions only
      applied to database connections operating within a single thread.
   <ol type="a">
   <li>The [sqlite3_enable_shared_cache()] interface now applies to all
       threads within a process, not to just the one thread in which it
       was run.</li>
   <li>The [sqlite3_soft_heap_limit()] interface now applies to all threads
       within a process, not to just the one thread in which it was run.</li>
   <li>The [sqlite3_release_memory()] interface will now attempt to reduce
       the memory usages across all database connections in all threads, not
       just connections in the thread where the interface is called.</li>
   <li>The [sqlite3_thread_cleanup()] interface has become a no-op.</li>
   </ol></li>
  <li>Restrictions on the use of the same database connection by multiple
      threads have been dropped.  It is now safe for
      multiple threads to use the same database connection at the same
      time.</li>
  <li>There is now a compile-time option that allows an application to
      define alternative malloc()/free() implementations without having
      to modify any core SQLite code.</li>
  <li>There is now a compile-time option that allows an application to
      define alternative mutex implementations without having
      to modify any core SQLite code.</li>
  </ol>
}
PARAGRAPH {
  Of these changes, only 1a and 2a through 2c are incompatibilities
  in any formal sense.
  But users who have previously made custom modifications to the
  SQLite source (for example to add a custom OS layer for embedded
  hardware) might find that these changes have a larger impact.
  On the other hand, an important goal of these changes is to make
  it much easier to customize SQLite for use on different operating
  systems. 
}

HEADING 1 {The OS Interface Layer}

PARAGRAPH {
  If your system defines a custom OS interface for SQLite or if you
  were using the undocumented <b>sqlite3_os_switch()</b>
  interface, then you will need to make modifications in order to
  upgrade to SQLite version 3.5.0.  This may seem painful at first
  glance.  But as you look more closely, you will probably discover
  that your changes are made smaller and easier to understand and manage
  by the new SQLite interface.  It is likely that your changes will
  now also work seamlessly with the SQLite amalgamation.  You will
  no longer need to make any changes to the code SQLite source code.
  All of your changes can be effected by application code and you can
  link against a standard, unmodified version of the SQLite amalgamation.
  Furthermore, the OS interface layer, which was formerly undocumented,
  is now an officially support interface for SQLite.  So you have
  some assurance that this will be a one-time change and that your
  new backend will continue to work in future versions of SQLite.
}

HEADING 2 {The Virtual File System Object}

PARAGRAPH {
  The new OS interface for SQLite is built around an object named
  [sqlite3_vfs].  The "vfs" standard for "Virtual File System".
  The sqlite3_vfs object is basically a structure containing pointers
  to functions that implement the primitive disk I/O operations that
  SQLite needs to perform in order to read and write databases.  
  In this article, we will often refer a sqlite3_vfs objects as a "VFS".
}

PARAGRAPH {
  SQLite is able to use multiple VFSes at the same time.  Each
  individual database connection is associated with just one VFS.
  But if you have multiple database connections, each connection
  can be associated with a different VFS.
}

PARAGRAPH {
  There is always a default VFS.
  The legacy interfaces [sqlite3_open()] and [sqlite3_open16()] always
  use the default VFS.
  The new interface for creating database connections,
  [sqlite3_open_v2()], allows you to specify which VFS you want to
  use by name.
}

HEADING 3 {Registering New VFS Objects}

PARAGRAPH {
  Standard builds of SQLite for unix or windows come with a single
  VFS named "unix" or "win32", as appropriate.  This one VFS is also 
  the default.  So if you are using the legacy open functions, everything
  will continue to operate as it has before.  The change is that an application
  now has the flexibility of adding new VFS modules to implement a
  customized OS layer.  The [sqlite3_vfs_register()] API can be used
  to tell SQLite about one or more application-defined VFS modules:
}

CODE {
int sqlite3_vfs_register(sqlite3_vfs*, int makeDflt);
}

PARAGRAPH {
  Applications can call sqlite3_vfs_register at any time, though of course
  a VFS needs to be registered before it can be used.  The first argument
  is a pointer to a customized VFS object that the application has prepared.
  The second argument is true to make the new VFS the default VFS so that
  it will be used by the legacy [sqlite3_open()] and [sqlite3_open16()] APIs.
  If the new VFS is not the default, then you will probably have to use
  the new [sqlite3_open_v2()] API to use it.  Note, however, that if
  a new VFS is the only VFS known to SQLite (if SQLite was compiled without
  its usual default VFS or if the pre-compiled default VFS was removed
  using [sqlite3_vfs_unregister()]) then the new VFS automatic becomes the
  default VFS regardless of the makeDflt argument to [sqlite3_vfs_register()].
}

PARAGRAPH {
  Standard builds include the default "unix" or "win32" VFSes.
  But if you use the -DOS_OTHER=1 compile-time option, then SQLite is
  built without a default VFS.  In that case, the application must
  register at least one VFS prior to calling [sqlite3_open()].
  This is the approach that embedded applications should use.
  Rather than modifying the SQLite source to to insert an alternative
  OS layer as was done in prior releases of SQLite, instead compile
  an unmodified SQLite source file (preferably the amalgamation)
  with the -DOS_OTHER=1 option, then invoke [sqlite3_vfs_register()]
  to define the interface to the underlying filesystem prior to
  creating any database connections.
}

HEADING 3 {Additional Control Over VFS Objects}

PARAGRAPH {
  The [sqlite3_vfs_unregister()] API is used to remove an existing
  VFS from the system.
}

CODE {
int sqlite3_vfs_unregister(sqlite3_vfs*);
}

PARAGRAPH {
  The [sqlite3_vfs_find()] API is used to locate a particular VFS
  by name.  Its prototype is as follows:
}

CODE {
sqlite3_vfs *sqlite3_vfs_find(const char *zVfsName);
}

PARAGRAPH {
  The argument is the symbolic name for the desired VFS.  If the
  argument is a NULL pointer, then the default VFS is returned.
  The function returns a pointer to the [sqlite3_vfs] object that
  implements the VFS.  Or it returns a NULL pointer if no object
  could be found that matched the search criteria.
}

HEADING 3 {Modifications Of Existing VFSes}

PARAGRAPH {
  Once a VFS has been registered, it should never be modified.  If
  a change in behavior is required, a new VFS should be registered.
  The application could, perhaps, use [sqlite3_vfs_find()] to locate
  the old VFS, make a copy of the old VFS into a new [sqlite3_vfs]
  object, make the desired modifications to the new VFS, unregister
  the old VFS, the register the new VFS in its place.  Existing
  database connections would continue to use the old VFS even after
  it is unregistered, but new database connections would use the
  new VFS.
}  

HEADING 3 {The VFS Object}

PARAGRAPH {
  A VFS object is an instance of the following structure:
}

CODE {
typedef struct sqlite3_vfs sqlite3_vfs;
struct sqlite3_vfs {
  int iVersion;            /* Structure version number */
  int szOsFile;            /* Size of subclassed sqlite3_file */
  int mxPathname;          /* Maximum file pathname length */
  sqlite3_vfs *pNext;      /* Next registered VFS */
  const char *zName;       /* Name of this virtual file system */
  void *pAppData;          /* Pointer to application-specific data */
  int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
               int flags, int *pOutFlags);
  int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
  int (*xAccess)(sqlite3_vfs*, const char *zName, int flags);
  int (*xGetTempName)(sqlite3_vfs*, char *zOut);
  int (*xFullPathname)(sqlite3_vfs*, const char *zName, char *zOut);
  void *(*xDlOpen)(sqlite3_vfs*, const char *zFilename);
  void (*xDlError)(sqlite3_vfs*, int nByte, char *zErrMsg);
  void *(*xDlSym)(sqlite3_vfs*,void*, const char *zSymbol);
  void (*xDlClose)(sqlite3_vfs*, void*);
  int (*xRandomness)(sqlite3_vfs*, int nByte, char *zOut);
  int (*xSleep)(sqlite3_vfs*, int microseconds);
  int (*xCurrentTime)(sqlite3_vfs*, double*);
  /* New fields may be appended in figure versions.  The iVersion
  ** value will increment whenever this happens. */
};
}

PARAGRAPH {
  To create a new VFS, an application fills in an instance of this
  structure with appropriate values and then calls [sqlite3_vfs_register()].
}

PARAGRAPH {
  The iVersion field of [sqlite3_vfs] should be 1 for SQLite version 3.5.0.
  This number may increase in future versions of SQLite if we have to
  modify the VFS object in some way.  We hope that this never happens,
  but the provision is made in case it does.
}

PARAGRAPH {
  The szOsFile field is the size in bytes of the structure that defines
  an open file: the [sqlite3_file] object.  This object will be described
  more fully below.  The point here is that each VFS implementation can
  define its own [sqlite3_file] object containing whatever information
  the VFS implementation needs to store about an open file.  SQLite needs
  to know how big this object is, however, in order to preallocate enough
  space to hold it.
}

PARAGRAPH {
  The mxPathname field is the maximum length of a file pathname that
  this VFS can use.  SQLite sometimes has to preallocate buffers of
  this size, so it should be as small as reasonably possible.  Some
  filesystems permit huge pathnames, but in practice pathnames rarely
  extend beyond 100 bytes or so.  You do not have to put the longest
  pathname that the underlying filesystem can handle here.  You only
  have to put the longest pathname that you want SQLite to be able to
  handle.  A few hundred is a good value in most cases.
}

PARAGRAPH {
  The pNext field is used internally by SQLite.  Specifically, SQLite
  uses this field to form a linked list of registered VFSes.
}

PARAGRAPH {
  The zName field is the symbolic name of the VFS.  This is the name 
  that the [sqlite3_vfs_find()] compares against when it is looking for
  a VFS.
}

PARAGRAPH {
  The pAppData pointer is unused by the SQLite core.  The pointer is
  available to store auxiliary information that a VFS information might
  want to carry around.
}

PARAGRAPH {
  The remaining fields of the [sqlite3_vfs] object all store pointer
  to functions that implement primitive operations.  We call these
  "methods".  The first methods, xOpen, is used to open files on
  the underlying storage media.  The result is an [sqlite3_file]
  object.  There are additional methods, defined by the [sqlite3_file]
  object itself that are used to read and write and close the file.
  The additional methods are detailed below.  The filename is in UTF-8.
  SQLite will guarantee that the zFilename string passed to
  xOpen() is a full pathname as generated by xFullPathname() and
  that the string will be valid and unchanged until xClose() is
  called.  So the [sqlite3_file] can store a pointer to the
   filename if it needs to remember the filename for some reason.
   The flags argument to xOpen() is a copy of the flags argument
   to sqlite3_open_v2().  If sqlite3_open() or sqlite3_open16()
   is used, then flags is [SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE].
   If xOpen() opens a file read-only then it sets *pOutFlags to
   include [SQLITE_OPEN_READONLY].  Other bits in *pOutFlags may be
   set.
   SQLite will also add one of the following flags to the xOpen()
   call, depending on the object being opened:
   <ul>
   <li>  [SQLITE_OPEN_MAIN_DB]
   <li>  [SQLITE_OPEN_MAIN_JOURNAL]
   <li>  [SQLITE_OPEN_TEMP_DB]
   <li>  [SQLITE_OPEN_TEMP_JOURNAL]
   <li>  [SQLITE_OPEN_TRANSIENT_DB]
   <li>  [SQLITE_OPEN_SUBJOURNAL]
   <li>  [SQLITE_OPEN_MASTER_JOURNAL]
   </ul>
   The file I/O implementation can use the object type flags to
   changes the way it deals with files.  For example, an application
   that does not care about crash recovery or rollback, might make
   the open of a journal file a no-op.  Writes to this journal are
   also a no-op.  Any attempt to read the journal returns [SQLITE_IOERR].
   Or the implementation might recognize the a database file will
   be doing page-aligned sector reads and writes in a random order
   and set up its I/O subsystem accordingly.
   SQLite might also add one of the following flags to the xOpen
   method:
   <ul>
   <li> [SQLITE_OPEN_DELETEONCLOSE]
   <li> [SQLITE_OPEN_EXCLUSIVE]
   </ul>
   The [SQLITE_OPEN_DELETEONCLOSE] flag means the file should be
   deleted when it is closed.  This will always be set for TEMP 
   databases and journals and for subjournals.  The 
   [SQLITE_OPEN_EXCLUSIVE] flag means the file should be opened
   for exclusive access.  This flag is set for all files except
   for the main database file.
   The [sqlite3_file] structure passed as the third argument to
   xOpen is allocated by the caller.  xOpen just fills it in.  The
   caller allocates a minimum of szOsFile bytes for the [sqlite3_file]
   structure.
}

PARAGRAPH {
  The differences between an [SQLITE_OPEN_TEMP_DB] database and an
  [SQLITE_OPEN_TRANSIENT_DB] database is this:  The [SQLITE_OPEN_TEMP_DB]
  is used for explicitly declared and named TEMP tables (using the
  CREATE TEMP TABLE syntax) or for named tables in a temporary database
  that is created by opening a database with a filename that is an empty
  string.  An [SQLITE_OPEN_TRANSIENT_DB] holds an database table that
  SQLite creates automatically in order to evaluate a subquery or
  ORDER BY or GROUP BY clause.  Both TEMP_DB and TRANSIENT_DB databases
  are private and are deleted automatically.  TEMP_DB databases last
  for the duration of the database connection.  TRANSIENT_DB databases
  last only for the duration of a single SQL statement.
}

PARAGRAPH {
  The xDelete method is used delete a file.  The name of the file is
  given in the second parameter.  The filename will be in UTF-8.
  The VFS must convert the filename into whatever character representation
  the underlying operating system expects.  If the syncDir parameter is
  true, then the xDelete method should not return until the change
  to the directory contents for the directory containing the
  deleted file have been synced to disk in order to insure that the
  file does not "reappear" if a power failure occurs soon after.
}

PARAGRAPH {
  The xAccess method is used to check for access permissions on a file.
  The filename will be UTF-8 encoded.  The flags argument will be
  [SQLITE_ACCESS_EXISTS] to check for the existence of the file,
  [SQLITE_ACCESS_READWRITE] to check to see if the file is both readable
  and writable, or [SQLITE_ACCESS_READ] to check to see if the file is
  at least readable.  The "file" named by the second parameter might
  be a directory or folder name.
}

PARAGRAPH {
  The xGetTempName method computes the name of a temporary file that
  SQLite can use.  The name should be written into the buffer given
  by the second parameter.  SQLite will size that buffer to hold
  at least mxPathname bytes.  The generated filename should be in UTF-8.
  To avoid security problems, the generated temporary filename should
  contain enough randomness to prevent an attacker from guessing the
  temporary filename in advance.
}

PARAGRAPH {
  The xFullPathname method is used to convert a relative pathname
  into a full pathname.  The resulting full pathname is written into
  the buffer provided by the third parameter.  SQLite will size the
  output buffer to at least mxPathname bytes.  Both the input and
  output names should be in UTF-8.
}

PARAGRAPH {
  The xDlOpen, xDlError, xDlSym, and xDlClose methods are all used for
  accessing shared libraries at run-time.  These methods may be omitted
  (and their pointers set to zero) if the library is compiled with
  SQLITE_OMIT_LOAD_EXTENSION or if the [sqlite3_enable_load_extension()]
  interface is never used to enable dynamic extension loading.  The
  xDlOpen method opens a shared library or DLL and returns a pointer to
  a handle.  NULL is returned if the open fails.  If the open fails,
  the xDlError method can be used to obtain a text error message.
  The message is written into the zErrMsg buffer of the third parameter
  which is at least nByte bytes in length.  The xDlSym returns a pointer
  to a symbol in the shared library.  The name of the symbol is given
  by the second parameter.  UTF-8 encoding is assumed.  If the symbol
  is not found a NULL pointer is returned.  The xDlClose routine closes
  the shared library.
}

PARAGRAPH {
  The xRandomness method is used exactly once to initialize the 
  pseudo-random number generator (PRNG) inside of SQLite.  Only
  the xRandomness method on the default VFS is used.  The xRandomness
  methods on other VFSes are never accessed by SQLite.
  The xRandomness routine requests that nByte bytes of randomness
  be written into zOut.  The routine returns the actual number of
  bytes of randomness obtained.  The quality of the randomness so obtained
  will determine the quality of the randomness generated by built-in 
  SQLite functions such as random() and randomblob().  SQLite also
  uses its PRNG to generate temporary file names..  On some platforms
  (ex: windows) SQLite assumes that temporary file names are unique
  without actually testing for collisions, so it is important to have
  good-quality randomness even if the random() and randomblob() 
  functions are never used.
}

PARAGRAPH {
  The xSleep method is used to suspend the calling thread for at
  least the number of microseconds given.  This method is used to
  implement the [sqlite3_sleep()] and [sqlite3_busy_timeout()] APIs.
  In the case of [sqlite3_sleep()] the xSleep method of the default
  VFS is always used.  If the underlying system does not have a
  microsecond resolution sleep capability, then the sleep time should
  be rounded up.  xSleep returns this rounded-up value.
}

PARAGRAPH {
  The xCurrentTime method finds the current time and date and writes
  the result as double-precision floating point value into pointer
  provided by the second parameter.  The time and date is in
  coordinated universal time (UTC) and is a fractional julian day number.
}

HEADING 3 {The Open File Object}

PARAGRAPH {
  The result of opening a file is an instance of an [sqlite3_file] object.
  The [sqlite3_file] object is an abstract base class defined as follows:
}

CODE {
typedef struct sqlite3_file sqlite3_file;
struct sqlite3_file {
  const struct sqlite3_io_methods *pMethods;
};
}

PARAGRAPH {
  Each VFS implementation will subclass the [sqlite3_file] by adding
  additional fields at the end to hold whatever information the VFS
  needs to know about an open file.  It does not matter what information
  is stored as long as the total size of the structure does not exceed
  the szOsFile value recorded in the [sqlite3_vfs] object.
}

PARAGRAPH {
  The [sqlite3_io_methods] object is a structure that contains pointers
  to methods for reading, writing, and otherwise dealing with files.
  This object is defined as follows:
}

CODE {
typedef struct sqlite3_io_methods sqlite3_io_methods;
struct sqlite3_io_methods {
  int iVersion;
  int (*xClose)(sqlite3_file*);
  int (*xRead)(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
  int (*xWrite)(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);
  int (*xTruncate)(sqlite3_file*, sqlite3_int64 size);
  int (*xSync)(sqlite3_file*, int flags);
  int (*xFileSize)(sqlite3_file*, sqlite3_int64 *pSize);
  int (*xLock)(sqlite3_file*, int);
  int (*xUnlock)(sqlite3_file*, int);
  int (*xCheckReservedLock)(sqlite3_file*);
  int (*xFileControl)(sqlite3_file*, int op, void *pArg);
  int (*xSectorSize)(sqlite3_file*);
  int (*xDeviceCharacteristics)(sqlite3_file*);
  /* Additional methods may be added in future releases */
};
}

PARAGRAPH {
  The iVersion field of [sqlite3_io_methods] is provided as insurance
  against future enhancements.  The iVersion value should always be
  1 for SQLite version 3.5.
}

PARAGRAPH {
  The xClose method closes the file.  The space for the [sqlite3_file]
  structure is deallocated by the caller.  But if the [sqlite3_file]
  contains pointers to other allocated memory or resources, those
  allocations should be released by the xClose method.
}

PARAGRAPH {
  The xRead method reads iAmt bytes from the file beginning at a byte
  offset to iOfst.  The data read is stored in the pointer of the
  second parameter.  xRead returns the [SQLITE_OK] on success,
  [SQLITE_IOERR_SHORT_READ] if it was not able to read the full number
  of bytes because it reached end-of-file, or [SQLITE_IOERR_READ] for
  any other error.
}

PARAGRAPH {
  The xWrite method writes iAmt bytes of data from the second parameter
  into the file beginning at an offset of iOfst bytes.  If the size of
  the file is less than iOfst bytes prior to the write, then xWrite should
  ensure that the file is extended with zeros up to iOfst bytes prior
  to beginning its write.  xWrite continues to extends the file as
  necessary so that the size of the file is at least iAmt+iOfst bytes 
  at the conclusion of the xWrite call.  The xWrite method returns
  [SQLITE_OK] on success.  If the write cannot complete because the
  underlying storage medium is full, then [SQLITE_FULL] is returned.
  [SQLITE_IOERR_WRITE] should be returned for any other error.
}

PARAGRAPH {
  The xTruncate method truncates a file to be nByte bytes in length.
  If the file is already nByte bytes or less in length then this
  method is a no-op.  The xTruncate method returns [SQLITE_OK] on
  success and [SQLITE_IOERR_TRUNCATE] if anything goes wrong.
}

PARAGRAPH {
  The xSync method is used to force previously written data out of
  operating system cache and into non-volatile memory.  The second
  parameter is usually [SQLITE_SYNC_NORMAL].  If the second parameter
  is [SQLITE_SYNC_FULL] then the xSync method should make sure that
  data has also been flushed through the disk controllers cache.
  The [SQLITE_SYNC_FULL] parameter is the equivalent of the F_FULLSYNC
  ioctl() on Mac OS X. The xSync method returns
  [SQLITE_OK] on success and [SQLITE_IOERR_FSYNC] if anything goes wrong.
}

PARAGRAPH {
  The xFileSize() method determines the current size of the file
  in bytes and writes that value into *pSize.  It returns [SQLITE_OK]
  on success and [SQLITE_IOERR_FSTAT] if something goes wrong.
}

PARAGRAPH {
  The xLock and xUnlock methods are used to set and clear file locks.
  SQLite supports five levels of file locks, in order:
  <ul>
  <li> [SQLITE_LOCK_NONE]
  <li> [SQLITE_LOCK_SHARED]
  <li> [SQLITE_LOCK_RESERVED]
  <li> [SQLITE_LOCK_PENDING]
  <li> [SQLITE_LOCK_EXCLUSIVE]
  </ul>
  The underlying implementation can support some subset of these locking
  levels as long as it meets the other requirements of this paragraph.
  The locking level is specified as the second argument to both xLock
  and xUnlock.  The xLock method increases the locking level to the
  specified locking level or higher.  The xUnlock method decreases the
  locking level to no lower than the level specified.  
  [SQLITE_LOCK_NONE] means that the file is unlocked.  [SQLITE_LOCK_SHARED]
  gives permission to read the file.  Multiple database connections can
  hold [SQLITE_LOCK_SHARED] at the same time.
  [SQLITE_LOCK_RESERVED] is like [SQLITE_LOCK_SHARED] in that its is permission
  to read the file.  But only a single connection can hold a reserved lock
  at any point in time.  The [SQLITE_LOCK_PENDING] is also permission to
  read the file.  Other connections can continue to read the file as well,
  but no other connection is allowed to escalate a lock from none to shared.
  [SQLITE_LOCK_EXCLUSIVE] is permission to write on the file.  Only a single
  connection can hold an exclusive lock and no other connection can hold
  any lock (other than "none") while one connection is hold an exclusive
  lock.  The xLock returns [SQLITE_OK] on success, [SQLITE_BUSY] if it
  is unable to obtain the lock, or [SQLITE_IOERR_RDLOCK] if something else
  goes wrong.  The xUnlock method returns [SQLITE_OK] on success and
  [SQLITE_IOERR_UNLOCK] for problems.
}

PARAGRAPH {
  The xCheckReservedLock method checks to see if another connection or
  another process is currently holding a reserved, pending, or exclusive
  lock on the file.  It returns true or false.
}

PARAGRAPH {
  The xFileControl() method is a generic interface that allows custom
  VFS implementations to directly control an open file using the
  (new and experimental)
  [sqlite3_file_control()] interface.  The second "op" argument
  is an integer opcode.   The third
  argument is a generic pointer which is intended to be a pointer
  to a structure that may contain arguments or space in which to
  write return values.  Potential uses for xFileControl() might be
  functions to enable blocking locks with timeouts, to change the
  locking strategy (for example to use dot-file locks), to inquire
  about the status of a lock, or to break stale locks.  The SQLite
  core reserves opcodes less than 100 for its own use. 
  A [SQLITE_FCNTL_LOCKSTATE | list of opcodes] less than 100 is available.
  Applications that define a custom xFileControl method should use opcodes 
  greater than 100 to avoid conflicts.
}

PARAGRAPH {
  The xSectorSize returns the "sector size" of the underlying
  non-volatile media.  A "sector" is defined as the smallest unit of
  storage that can be written without disturbing adjacent storage.
  On a disk drive the "sector size" has until recently been 512 bytes,
  though there is a push to increase this value to 4KiB.  SQLite needs
  to know the sector size so that it can write a full sector at a
  time, and thus avoid corrupting adjacent storage space if a power
  lose occurs in the middle of a write.
}

PARAGRAPH {
  The xDeviceCharacteristics method returns an integer bit vector that
  defines any special properties that the underlying storage medium might
  have that SQLite can use to increase performance.  The allowed return
  is the bit-wise OR of the following values:
  <ul>
  <li> [SQLITE_IOCAP_ATOMIC]
  <li> [SQLITE_IOCAP_ATOMIC512]
  <li> [SQLITE_IOCAP_ATOMIC1K]
  <li> [SQLITE_IOCAP_ATOMIC2K]
  <li> [SQLITE_IOCAP_ATOMIC4K]
  <li> [SQLITE_IOCAP_ATOMIC8K]
  <li> [SQLITE_IOCAP_ATOMIC16K]
  <li> [SQLITE_IOCAP_ATOMIC32K]
  <li> [SQLITE_IOCAP_ATOMIC64K]
  <li> [SQLITE_IOCAP_SAFE_APPEND]
  <li> [SQLITE_IOCAP_SEQUENTIAL]
  </ul>
  The [SQLITE_IOCAP_ATOMIC] bit means that all writes to this device are
  atomic in the sense that either the entire write occurs or none of it
  occurs.  The other 
  [SQLITE_IOCAP_ATOMIC | SQLITE_IOCAP_ATOMIC<i>nnn</i>] values indicate that
  writes of aligned blocks of the indicated size are atomic.
  [SQLITE_IOCAP_SAFE_APPEND] means that when extending a file with new
  data, the new data is written first and then the file size is updated.
  So if a power failure occurs, there is no chance that the file might have
  been extended with randomness.  The [SQLITE_IOCAP_SEQUENTIAL] bit means
  that all writes occur in the order that they are issued and are not
  reordered by the underlying file system.
}

HEADING 3 {Checklist For Constructing A New VFS}

PARAGRAPH {
  The preceding paragraphs contain a lot of information.
  To ease the task of constructing
  a new VFS for SQLite we offer the following implementation checklist:
}

PARAGRAPH {
  <ol>
  <li> Define an appropriate subclass of the [sqlite3_file] object.
  <li> Implement the methods required by the [sqlite3_io_methods] object.
  <li> Create a static and 
       constant [sqlite3_io_methods] object containing pointers
       to the methods from the previous step.
  <li> Implement the xOpen method that opens a file and populates an
       [sqlite3_file] object, including setting pMethods to
       point to the [sqlite3_io_methods] object from the previous step.
  <li> Implement the other methods required by [sqlite3_vfs].
  <li> Define a static (but not constant) [sqlite3_vfs] structure that
       contains pointers to the xOpen method and the other methods and
       which contains the appropriate values for iVersion, szOsFile,
       mxPathname, zName, and pAppData.
  <li> Implement a procedure that calls [sqlite3_vfs_register()] and
       passes it a pointer to the [sqlite3_vfs] structure from the previous
       step.  This procedure is probably the only exported symbol in the
       source file that implements your VFS.
  </ol>
}

PARAGRAPH {
  Within your application, call the procedure implemented in the last
  step above as part of your initialization process before any
  database connections are opened.  
}

HEADING 1 {The Memory Allocation Subsystem}

PARAGRAPH {
  Beginning with version 3.5, SQLite obtains all of the heap memory it
  needs using the routines [sqlite3_malloc()], [sqlite3_free()], and
  [sqlite3_realloc()].  These routines have existed in prior versions
  of SQLite, but SQLite has previously bypassed these routines and used
  its own memory allocator.  This all changes in version 3.5.0.
}

PARAGRAPH {
  The SQLite source tree actually contains multiple versions of the
  memory allocator.  The default high-speed version found in the
  "mem1.c" source file is used for most builds.  But if the SQLITE_MEMDEBUG
  flag is enabled, a separate memory allocator the "mem2.c" source file
  is used instead.  The mem2.c allocator implements lots of hooks to
  do error checking and to simulate memory allocation failures for testing
  purposes.  Both of these allocators use the malloc()/free() implementation
  in the standard C library.
}

PARAGRAPH {
  Applications are not required to use either of these standard memory
  allocators.  If SQLite is compiled with SQLITE_OMIT_MEMORY_ALLOCATION
  then no implementation for the [sqlite3_malloc()], [sqlite3_realloc()],
  and [sqlite3_free()] functions is provided.  Instead, the application
  that links against SQLite must provide its own implementation of these
  functions.  The application provided memory allocator is not required
  to use the malloc()/free() implementation in the standard C library.
  An embedded application might provide an alternative memory allocator
  that uses memory for a fixed memory pool set aside for the exclusive
  use of SQLite, for example.
}

PARAGRAPH {
  Applications that implement their own memory allocator must provide
  implementation for the usual three allocation functions 
  [sqlite3_malloc()], [sqlite3_realloc()], and [sqlite3_free()].
  And they must also implement a fourth function:
}

CODE {
int sqlite3_memory_alarm(
  void(*xCallback)(void *pArg, sqlite3_int64 used, int N),
  void *pArg,
  sqlite3_int64 iThreshold
);
}

PARAGRAPH {
 The [sqlite3_memory_alarm] routine is used to register
 a callback on memory allocation events.
 This routine registers or clears a callbacks that fires when
 the amount of memory allocated exceeds iThreshold.  Only
 a single callback can be registered at a time.  Each call
 to [sqlite3_memory_alarm()] overwrites the previous callback.
 The callback is disabled by setting xCallback to a NULL
 pointer.
}

PARAGRAPH {
 The parameters to the callback are the pArg value, the 
 amount of memory currently in use, and the size of the
 allocation that provoked the callback.  The callback will
 presumably invoke [sqlite3_free()] to free up memory space.
 The callback may invoke [sqlite3_malloc()] or [sqlite3_realloc()]
 but if it does, no additional callbacks will be invoked by
 the recursive calls.
}

PARAGRAPH {
 The [sqlite3_soft_heap_limit()] interface works by registering
 a memory alarm at the soft heap limit and invoking 
 [sqlite3_release_memory()] in the alarm callback.  Application
 programs should not attempt to use the [sqlite3_memory_alarm()]
 interface because doing so will interfere with the
 [sqlite3_soft_heap_limit()] module.  This interface is exposed
 only so that applications can provide their own
 alternative implementation when the SQLite core is
 compiled with SQLITE_OMIT_MEMORY_ALLOCATION.
}

PARAGRAPH {
  The built-in memory allocators in SQLite also provide the following
  additional interfaces:
}

CODE {
sqlite3_int64 sqlite3_memory_used(void);
sqlite3_int64 sqlite3_memory_highwater(int resetFlag);
}

PARAGRAPH {
  These interfaces can be used by an application to monitor how
  much memory SQLite is using.  The [sqlite3_memory_used()] routine
  returns the number of bytes of memory currently in use and the
  [sqlite3_memory_highwater()] returns the maximum instantaneous
  memory usage.  Neither routine includes the overhead associated
  with the memory allocator.  These routines are provided for use
  by the application.  SQLite never invokes them itself.  So if
  the application is providing its own memory allocation subsystem,
  it can omit these interfaces if desired.
}

HEADING 1 {The Mutex Subsystem}

PARAGRAPH {
  SQLite has always been threadsafe in the sense that it is safe to
  use different SQLite database connections in different threads at the
  same time.  The constraint was that the same database connection
  could not be used in two separate threads at once.  SQLite version 3.5.0
  relaxes this constraint. 
}

PARAGRAPH {
  In order to allow multiple threads to use the same database connection
  at the same time, SQLite must make extensive use of mutexes.  And for
  this reason a new mutex subsystem as been added.  The mutex subsystem
  as the following interface:
}

CODE {
sqlite3_mutex *sqlite3_mutex_alloc(int);
void sqlite3_mutex_free(sqlite3_mutex*);
void sqlite3_mutex_enter(sqlite3_mutex*);
int sqlite3_mutex_try(sqlite3_mutex*);
void sqlite3_mutex_leave(sqlite3_mutex*);
}

PARAGRAPH {
  Though these routines exist for the use of the SQLite core, 
  application code is free to use these routines as well, if desired.
  A mutex is an [sqlite3_mutex] object.  The [sqlite3_mutex_alloc()]
  routine allocates a new mutex object and returns a pointer to it.
  The argument to [sqlite3_mutex_alloc()] should be 
  [SQLITE_MUTEX_FAST] or [SQLITE_MUTEX_RECURSIVE] for non-recursive
  and recursive mutexes, respectively.  If the underlying system does
  not provide non-recursive mutexes, then a recursive mutex can be
  substituted in that case.  The argument to [sqlite3_mutex_alloc()]
  can also be a constant designating one of several static mutexes:
  <ul>
  <li>  [SQLITE_MUTEX_STATIC_MASTER]
  <li>  [SQLITE_MUTEX_STATIC_MEM]
  <li>  [SQLITE_MUTEX_STATIC_MEM2]
  <li>  [SQLITE_MUTEX_STATIC_PRNG]
  <li>  [SQLITE_MUTEX_STATIC_LRU]
  </ul>
  These static mutexes are reserved for use internally by SQLite
  and should not be used by the application.  The static mutexes
  are all non-recursive.
}

PARAGRAPH {
  The [sqlite3_mutex_free()] routine should be used to deallocate
  a non-static mutex.  If a static mutex is passed to this routine
  then the behavior is undefined.
}

PARAGRAPH {
  The [sqlite3_mutex_enter()] attempts to enter the mutex and blocks
  if another threads is already there.  [sqlite3_mutex_try()] attempts
  to enter and returns [SQLITE_OK] on success or [SQLITE_BUSY] if another
  thread is already there.  [sqlite3_mutex_leave()] exits a mutex.
  The mutex is held until the number of exits matches the number of
  entrances.  If [sqlite3_mutex_leave()] is called on a mutex that 
  the thread is not currently holding, then the behavior is undefined.
  If any routine is called for a deallocated mutex, then the behavior
  is undefined.
}

PARAGRAPH {
  The SQLite source code provides multiple implementations of these
  APIs, suitable for varying environments.  If SQLite is compiled with
  the SQLITE_THREADSAFE=0 flag then a no-op mutex implementation that 
  is fast but does no real mutual exclusion is provided.  That 
  implementation is suitable for use in single-threaded applications
  or applications that only use SQLite in a single thread.  Other
  real mutex implementations are provided based on the underlying
  operating system.
}

PARAGRAPH {
  Embedded applications may wish to provide their own mutex implementation.
  If SQLite is compiled with the -DSQLITE_MUTEX_APPDEF=1 compile-time flag
  then the SQLite core provides no mutex subsystem and a mutex subsystem
  that matches the interface described above must be provided by the
  application that links against SQLite.
}

HEADING 1 {Other Interface Changes}

PARAGRAPH {
  Version 3.5.0 of SQLite changes the behavior of a few APIs in ways
  that are technically incompatible.  However, these APIs are seldom
  used and even when they are used it is difficult to imagine a
  scenario where the change might break something.  The changes
  actually makes these interface much more useful and powerful.
}

PARAGRAPH {
  Prior to version 3.5.0, the [sqlite3_enable_shared_cache()] API
  would enable and disable the shared cache feature for all connections
  within a single thread - the same thread from which the 
  sqlite3_enable_shared_cache() routine was called.  Database connections
  that used the shared cache were restricted to running in the same
  thread in which they were opened.  Beginning with version 3.5.0,
  the sqlite3_enable_shared_cache() applies to all database connections
  in all threads within the process.  Now database connections running
  in separate threads can share a cache.  And database connections that
  use shared cache can migrate from one thread to another.
}

PARAGRAPH {
  Prior to version 3.5.0 the [sqlite3_soft_heap_limit()] set an upper
  bound on heap memory usage for all database connections within a
  single thread.  Each thread could have its own heap limit.  Beginning
  in version 3.5.0, there is a single heap limit for the entire process.
  This seems more restrictive (one limit as opposed to many) but in
  practice it is what most users want.
}

PARAGRAPH {
  Prior to version 3.5.0 the [sqlite3_release_memory()] function would
  try to reclaim memory from all database connections in the same thread
  as the sqlite3_release_memory() call.  Beginning with version 3.5.0,
  the sqlite3_release_memory() function will attempt to reclaim memory
  from all database connections in all threads.
}

HEADING 1 {Summary}

PARAGRAPH {
  The transition from SQLite version 3.4.2 to 3.5.0 is a major change.
  Every source code file in the SQLite core had to be modified, some
  extensively.  And the change introduced some minor incompatibilities
  in the C interface.  But we feel that the benefits of the transition
  from 3.4.2 to 3.5.0 far outweigh the pain of porting.  The new
  VFS layer is now well-defined and stable and should simplify future
  customizations.  The VFS layer, and the separable memory allocator
  and mutex subsystems allow a standard SQLite source code amalgamation
  to be used in an embedded project without change, greatly simplifying
  configuration management.  And the resulting system is much more
  tolerant of highly threaded designs.
}