mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improved comments on the server and asynchronous I/O demo programs. (CVS 2909)

Browse Source 
FossilOrigin-Name: c0f47ccbc915f20d56f393383c21b4026785e6a5
This commit is contained in:
drh 2006-01-10 20:01:18 +00:00
parent ee32e0aa27
commit fe0f75bb1f
4 changed files with 137 additions and 50 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
manifest
View File

@ -1,5 +1,5 @@
C More\spedantic\schanges\sto\scomments\sin\sVDBE.\s\sNo\schanges\sto\scode.\s\sTicket\s#1596.\s(CVS\s2908)
D 2006-01-10T19:45:49
C Improved\scomments\son\sthe\sserver\sand\sasynchronous\sI/O\sdemo\sprograms.\s(CVS\s2909)
D 2006-01-10T20:01:19
F Makefile.in ab3ffd8d469cef4477257169b82810030a6bb967
F Makefile.linux-gcc aee18d8a05546dcf1888bd4547e442008a49a092
F README 9c4e2d6706bdcc3efdd773ce752a8cdab4f90028
@ -67,7 +67,7 @@ F src/prepare.c 3283bb65b4b217a092c9cbf65014774e6c3a142d
F src/printf.c f47a2f4b5387cd2ebb12e9117a1a5d6bd9a2b812
F src/random.c d40f8d356cecbd351ccfab6eaedd7ec1b54f5261
F src/select.c 579cfdd250c5598de7c867134f7d35a2099b1dcc
F src/server.c 42a2bd02eec5018098a96e08f7a923f4965a2b1d
F src/server.c 519e308651e30102dd3d1f4053ac64c14267e44c
F src/shell.c 66b073375efbdee19045e7e0cd38b85f9aff71da
F src/sqlite.h.in 821b93f918d126c54d9a91fc928434945655edc3
F src/sqliteInt.h d7584dc5b8e15f1732a195ece9e93049ccde35fa
@ -80,7 +80,7 @@ F src/test4.c a8fd681e139e1c61f22a77d07fc3a99cb28fff3f
F src/test5.c 7162f8526affb771c4ed256826eee7bb9eca265f
F src/test6.c 74d91b487c68154156eded457925d96aa2a3fdbb
F src/test7.c bfe36c62cae189509660acfeeb891ffb9da8ef0c
F src/test_async.c 9733deb7fefa18a3596e5234c1ef05b4685c6ad7
F src/test_async.c 6776f5027ca6378c116ff5ccc2fe41b908e33772
F src/tokenize.c 196486012c871cdcad6cc84a820cc988603f1b9d
F src/trigger.c 883b5f3b97137fbe417e3337c3fa20ac8e9c1ae5
F src/update.c cd8ad5bb1a29f2056347481308fca4a59f2f4764
@ -340,7 +340,7 @@ F www/tclsqlite.tcl bb0d1357328a42b1993d78573e587c6dcbc964b9
F www/vdbe.tcl 87a31ace769f20d3627a64fa1fade7fed47b90d0
F www/version3.tcl a99cf5f6d8bd4d5537584a2b342f0fb9fa601d8b
F www/whentouse.tcl 97e2b5cd296f7d8057e11f44427dea8a4c2db513
P 511ac9db12ad84bb02d84568b75fc65cef661e88
R dc7dbd20c2b4f7e0aa62e74e583aeae0
P 1cf6855430352ffbf921a977186345d7272fe272
R c77d71bb3bbb59cc1b67cf59dbab5a91
U drh
Z 55bfd4e79be2428d625d90a19139bf0e
Z 85b6b185878e0427b46314ea83850303

2
manifest.uuid
View File

@ -1 +1 @@
1cf6855430352ffbf921a977186345d7272fe272
c0f47ccbc915f20d56f393383c21b4026785e6a5

5
src/server.c
View File

@ -62,8 +62,7 @@
**
** Note: The extra features of version 3.3.0 described by points (2)
** through (4) above are only available if you compile without the
** option -DSQLITE_OMIT_SHARED_CACHE. For reasons of backwards
** compatibility, SQLite is compile with this option by default.
** option -DSQLITE_OMIT_SHARED_CACHE.
**
** Here is how the client/server approach works: The database server
** thread is started on this procedure:
@ -256,7 +255,7 @@ static void sendToServer(SqlMessage *pMsg){
** sqlite3_close
**
** Clients should use the following client-side routines instead of
** the core routines.
** the core routines above.
**
** sqlite3_client_open
** sqlite3_client_prepare

166
src/test_async.c
View File

@ -11,33 +11,100 @@
*************************************************************************
**
** This file contains an example implementation of an asynchronous IO
** backend for SQLite. It is used to test that the concept of asynchronous
** IO in SQLite is valid.
** backend for SQLite.
**
** WHAT IS ASYNCHRONOUS I/O?
**
** With asynchronous I/O, write requests are handled by a separate thread
** running in the background. This means that the thread that initiates
** a database write does not have to wait for (sometimes slow) disk I/O
** to occur. The write seems to happen very quickly, though in reality
** it is happening at its usual slow pace in the background.
**
** Asynchronous I/O appears to give better responsiveness, but at a price.
** You lose the Durable property. With the default I/O backend of SQLite,
** once a write completes, you know that the information you wrote is
** safely on disk. With the asynchronous I/O, this is no the case. If
** your program crashes or if you take a power lose after the database
** write but before the asynchronous write thread has completed, then the
** database change might never make it to disk and the next user of the
** database might not see your change.
**
** You lose Durability with asynchronous I/O, but you still retain the
** other parts of ACID: Atomic, Consistent, and Isolated. Many
** appliations get along fine without the Durablity.
**
** HOW IT WORKS
**
** Asynchronous I/O works by overloading the OS-layer disk I/O routines
** with modified versions that store the data to be written in queue of
** pending write operations. Look at the asyncEnable() subroutine to see
** how overloading works. Six os-layer routines are overloaded:
**
** sqlite3OsOpenReadWrite;
** sqlite3OsOpenReadOnly;
** sqlite3OsOpenExclusive;
** sqlite3OsDelete;
** sqlite3OsFileExists;
** sqlite3OsSyncDirectory;
**
** The original implementations of these routines are saved and are
** used by the writer thread to do the real I/O. The substitute
** implementations typically put the I/O operation on a queue
** to be handled later by the writer thread, though read operations
** must be handled right away, obviously.
**
** Asynchronous I/O is disabled by setting the os-layer interface routines
** back to their original values.
**
** LIMITATIONS
**
** This demonstration code is deliberately kept simple in order to keep
** the main ideas clear and easy to understand. Real applications that
** want to do asynchronous I/O might want to add additional capabilities.
** For example, in this demonstration if writes are happening at a steady
** stream that exceeds the I/O capability of the background writer thread,
** the queue of pending write operations will grow without bound until we
** run out of memory. Users of this technique may want to keep track of
** the quantity of pending writes and stop accepting new write requests
** when the buffer gets to be too big.
*/
#include "sqliteInt.h"
#include "os.h"
#include <tcl.h>
/* If the THREADSAFE macro is not set, assume that it is turned off. */
#ifndef THREADSAFE
# define THREADSAFE 0
#endif
/*
** This test uses pthreads and hence only works on unix and with
** a threadsafe build of SQLite.
** a threadsafe build of SQLite. It also requires that the redefinable
** I/O feature of SQLite be turned on. This feature is turned off by
** default. If a required element is missing, almost all of the code
** in this file is commented out.
*/
#if OS_UNIX && THREADSAFE && defined(SQLITE_ENABLE_REDEF_IO)
/*
** This demo uses pthreads. If you do not have a pthreads implementation
** for your operating system, you will need to recode the threading
** logic.
*/
#include <pthread.h>
#include <sched.h>
/* Useful macros used in several places */
#define MIN(x,y) ((x)<(y)?(x):(y))
#define MAX(x,y) ((x)>(y)?(x):(y))
/* Forward references */
typedef struct AsyncWrite AsyncWrite;
typedef struct AsyncFile AsyncFile;
/* Enable for debugging */
#if 0
# define TRACE(X,Y) \
fprintf(stderr,"THRD=%d: ", (int)pthread_self()); \
@ -46,11 +113,6 @@ typedef struct AsyncFile AsyncFile;
# define TRACE(X,Y) /* noop */
#endif
/*
** TODO:
** * File locks...
*/
/*
** THREAD SAFETY NOTES
**
@ -62,7 +124,7 @@ typedef struct AsyncFile AsyncFile;
** * The file handles from the underlying system are assumed not to
** be thread safe.
**
** * See the last paragraph under "sqlite3_async_flush() Threads" for
** * See the last two paragraphs under "The Writer Thread" for
** an assumption to do with file-handle synchronization by the Os.
**
** File system operations (invoked by SQLite thread):
@ -74,39 +136,47 @@ typedef struct AsyncFile AsyncFile;
**
** File handle operations (invoked by SQLite thread):
**
** The following operations add an entry to the global write-op list. They
** prepare the entry, acquire the mutex momentarily while list pointers are
** manipulated to insert the new entry, and release the mutex.
**
** asyncWrite, asyncClose, asyncTruncate, asyncSync,
** asyncSetFullSync, asyncOpenDirectory.
**
** Read operations. Both of these read from both the underlying file and
** the write-op list. So we grab the mutex for the whole call (even
** while performing a blocking read on the file).
** The operations above add an entry to the global write-op list. They
** prepare the entry, acquire the async.queueMutex momentarily while
** list pointers are manipulated to insert the new entry, then release
** the mutex and signal the writer thread to wake up in case it happens
** to be asleep.
**
**
** asyncRead, asyncFileSize.
**
** Read operations. Both of these read from both the underlying file
** first then adjust their result based on pending writes in the
** write-op queue. So async.queueMutex is held for the duration
** of these operations to prevent other threads from changing the
** queue in mid operation.
**
**
** asyncLock, asyncUnlock, asyncLockState, asyncCheckReservedLock
**
** These locking primitives become no-ops. Files are always opened for
** exclusive access when using this IO backend:
**
** asyncLock, asyncUnlock, asyncLockState, asyncCheckReservedLock
** exclusive access when using this IO backend.
**
**
** asyncFileHandle.
**
** The sqlite3OsFileHandle() function is currently only used when
** debugging the pager module. Unless sqlite3OsClose() is called on the
** file (shouldn't be possible for other reasons), the underlying
** implementations are safe to call without grabbing any mutex. So we just
** go ahead and call it no matter what any other thread is doing.
** go ahead and call it no matter what any other threads are doing.
**
** asyncFileHandle.
**
** asyncSeek.
**
** Calling this method just manipulates the AsyncFile.iOffset variable.
** Since this variable is never accessed by writer thread, this
** function does not require the mutex. Actual calls to OsSeek() take
** place just before OsWrite() or OsRead(), which are always protected by
** the mutex.
**
** asyncSeek.
**
** The writer thread:
**
@ -123,7 +193,7 @@ typedef struct AsyncFile AsyncFile;
** The async.queueMutex is always held during the <write-op list is
** not empty> test, and when the entry is removed from the head
** of the write-op list. Sometimes it is held for the interim
** period (while the IO is performed), and sometimes it is
** period (while the IO is performed), and sometimes it is
** relinquished. It is relinquished if (a) the IO op is an
** ASYNC_CLOSE or (b) when the file handle was opened, two of
** the underlying systems handles were opened on the same
@ -186,6 +256,9 @@ static struct TestAsyncStaticData {
#define ASYNC_SYNCDIRECTORY 9
/*
** Entries on the write-op queue are instances of the AsyncWrite
** structure, defined here.
**
** The interpretation of the iOffset and nByte variables varies depending
** on the value of AsyncWrite.op:
**
@ -248,15 +321,19 @@ struct AsyncFile {
/*
** Add an entry to the end of the global write-op list. pWrite should point
** to an AsyncWrite structure allocated using sqliteMalloc(). A future call
** to sqlite3_async_flush() is responsible for calling sqliteFree().
** to an AsyncWrite structure allocated using sqlite3OsMalloc(). The writer
** thread will call sqlite3OsFree() to free the structure after the specified
** operation has been completed.
**
** Once an AsyncWrite structure has been added to the list, it must not be
** read or modified by the caller (in case another thread calls
** sqlite3_async_flush() ).
** Once an AsyncWrite structure has been added to the list, it becomes the
** property of the writer thread and must not be read or modified by the
** caller.
*/
static void addAsyncWrite(AsyncWrite *pWrite){
/* We must hold the queue mutex in order to modify the queue pointers */
pthread_mutex_lock(&async.queueMutex);
/* Add the record to the end of the write-op queue */
assert( !pWrite->pNext );
if( async.pQueueLast ){
assert( async.pQueueFirst );
@ -266,7 +343,12 @@ static void addAsyncWrite(AsyncWrite *pWrite){
}
async.pQueueLast = pWrite;
TRACE("PUSH %p\n", pWrite);
/* Drop the queue mutex */
pthread_mutex_unlock(&async.queueMutex);
/* The writer thread might have been idle because there was nothing
** on the write-op queue for it to do. So wake it up. */
pthread_cond_signal(&async.queueSignal);
}
@ -358,8 +440,7 @@ static void asyncSetFullSync(OsFile *id, int value){
/*
** Read data from the file. First we read from the filesystem, then adjust
** the contents of the buffer based on ASYNC_WRITE operations in the
** write-op queue. Todo: Do we need to think about ASYNC_TRUNCATE in
** this method as well?
** write-op queue.
**
** This method holds the mutex from start to finish.
*/
@ -478,6 +559,10 @@ static int asyncFileHandle(OsFile *id){
return sqlite3OsFileHandle(((AsyncFile *)id)->pBaseRead);
}
/*
** No file locking occurs with this version of the asynchronous backend.
** So the locking routines are no-ops.
*/
static int asyncLock(OsFile *id, int lockType){
return SQLITE_OK;
}
@ -502,8 +587,8 @@ static int asyncLockState(OsFile *id){
/*
** The following variables hold pointers to the original versions of
** certain OS-layer interface routines - routines that this module
** overrides.
** OS-layer interface routines that are overloaded in order to create
** the asynchronous I/O backend.
*/
static int (*xOrigOpenReadWrite)(const char*, OsFile**, int*) = 0;
static int (*xOrigOpenExclusive)(const char*, OsFile**, int) = 0;
@ -512,12 +597,15 @@ static int (*xOrigDelete)(const char*) = 0;
static int (*xOrigFileExists)(const char*) = 0;
static int (*xOrigSyncDirectory)(const char*) = 0;
/*
** This routine does most of the work of opening a file and building
** the OsFile structure.
*/
static int asyncOpenFile(
const char *zName,
OsFile **pFile,
OsFile *pBaseRead,
int openSecondFile
const char *zName, /* The name of the file to be opened */
OsFile **pFile, /* Put the OsFile structure here */
OsFile *pBaseRead, /* The real OsFile from the real I/O routine */
int openForWriting /* Open a second file handle for writing if true */
){
int rc;
AsyncFile *p;
@ -540,7 +628,7 @@ static int asyncOpenFile(
asyncCheckReservedLock
};
if( openSecondFile && SQLITE_ASYNC_TWO_FILEHANDLES ){
if( openForWriting && SQLITE_ASYNC_TWO_FILEHANDLES ){
int dummy;
rc = xOrigOpenReadWrite(zName, &pBaseWrite, &dummy);
if( rc!=SQLITE_OK ){