mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add tests for another application writing the database while an ota update is ongoing.

Browse Source 
FossilOrigin-Name: 2402baa0027ca65e9a5106b8b9c4e10f40d2cbc3
This commit is contained in:
dan 2014-10-21 18:09:58 +00:00
parent ae0151c5a7
commit 5e3fc6e99f
5 changed files with 180 additions and 33 deletions

30
ext/ota/README.txt

@ -19,11 +19,11 @@ SQLite Hacks
* All non-temporary triggers are disabled.
2) PRAGMA pager_ota_mode:
2) PRAGMA pager_ota_mode=1:
This pragma sets a flag on the pager associated with the main database only. In
a zipvfs system, this pragma is intercepted by zipvfs and the flag is set on
the lower level pager only.
This pragma sets a flag on the pager associated with the main database only.
In a zipvfs system, this pragma is intercepted by zipvfs and the flag is set
on the lower level pager only.
The flag can only be set when there is no open transaction and the pager does
not already have an open WAL file. Attempting to do so is an error.
@ -36,13 +36,13 @@ SQLite Hacks
Otherwise, if no WAL file or flags are found, the pager opens the *-oal file
and uses it as a write-ahead-log with the *-shm data stored in heap-memory.
The 8-bytes of "salt" at teh start of an *-oal file is a copy of the 8 bytes
The 8-bytes of "salt" at the start of an *-oal file is a copy of the 8 bytes
starting at offset 24 of the database file header (the change counter and the
number of pages in the file). If the *-oal file already exists when it is
opened, SQLite checks that the salt still matches the database header fields.
If not, it concludes that the database file has been written by a rollback-mode
client since the *-oal wa created and an SQLITE_BUSY_SNAPSHOT error is
returned. No read-transaction can be opened in this case.
If not, it concludes that the database file has been written by a
rollback-mode client since the *-oal wa created and an SQLITE_BUSY_SNAPSHOT
error is returned. No read-transaction can be opened in this case.
A pager with the pager_ota_mode flag set never runs a checkpoint.
@ -51,14 +51,26 @@ SQLite Hacks
pager_ota_mode connections. If two or more such connections attempt to write
simultaneously, the results are undefined.
3) PRAGMA pager_ota_mode=2:
3) sqlite3_index_writer()
The pager_ota_mode pragma may also be set to 2 if the main database is open
in WAL mode. This prevents SQLite from checkpointing the wal file as part
of sqlite3_close().
The effects of setting pager_ota_mode=2 if the db is not in WAL mode are
undefined.
4) sqlite3_index_writer()
This new API function is used to create VMs that can insert or delete entries
from individual index b-trees within the database. The VMs apply affinities
and check that UNIQUE constraints are not violated before updating index
b-trees.
5) sqlite3_ckpt_open/step/close()
API for performing (and resuming) incremental checkpoints.
The OTA extension
-----------------

128
ext/ota/ota6.test Normal file

@ -0,0 +1,128 @@
# 2014 October 21
#
# The author disclaims copyright to this source code. In place of
# a legal notice, here is a blessing:
#
# May you do good and not evil.
# May you find forgiveness for yourself and forgive others.
# May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains tests for the OTA module. Specifically, it tests the
# outcome of some other client writing to the database while an OTA update
# is being applied.
if {![info exists testdir]} {
set testdir [file join [file dirname [info script]] .. .. test]
}
source $testdir/tester.tcl
set ::testprefix ota6
proc setup_test {} {
reset_db
execsql {
CREATE TABLE t1(a INTEGER PRIMARY KEY, b UNIQUE);
CREATE TABLE t2(a INTEGER PRIMARY KEY, b UNIQUE);
CREATE TABLE t3(a INTEGER PRIMARY KEY, b UNIQUE);
}
db close
forcedelete ota.db
sqlite3 ota ota.db
ota eval {
CREATE TABLE data_t1(a, b, ota_control);
CREATE TABLE data_t2(a, b, ota_control);
CREATE TABLE data_t3(a, b, ota_control);
INSERT INTO data_t1 VALUES(1, 't1', 0);
INSERT INTO data_t2 VALUES(2, 't2', 0);
INSERT INTO data_t3 VALUES(3, 't3', 0);
}
ota close
}
# Test the outcome of some other client writing the db while the *-oal
# file is being generated. Once this has happened, the update cannot be
# progressed.
#
for {set nStep 1} {$nStep < 7} {incr nStep} {
do_test 1.$nStep.1 {
setup_test
sqlite3ota ota test.db ota.db
for {set i 0} {$i<$nStep} {incr i} {ota step}
ota close
sqlite3 db test.db
execsql { INSERT INTO t1 VALUES(5, 'hello') }
sqlite3ota ota test.db ota.db
ota step
} {SQLITE_BUSY}
do_test 1.$nStep.2 {
ota step
} {SQLITE_BUSY}
do_test 1.$nStep.3 {
list [file exists test.db-oal] [file exists test.db-wal]
} {1 0}
do_test 1.$nStep.4 {
list [catch { ota close } msg] $msg
} {1 {SQLITE_BUSY - database is locked}}
}
for {set nStep 7} {$nStep < 8} {incr nStep} {
do_test 1.$nStep.1 {
setup_test
sqlite3ota ota test.db ota.db
for {set i 0} {$i<$nStep} {incr i} {ota step}
ota close
sqlite3 db test.db
execsql { INSERT INTO t1 VALUES(5, 'hello') }
sqlite3ota ota test.db ota.db
ota step
} {SQLITE_OK}
do_test 1.$nStep.2 {
ota step
} {SQLITE_OK}
do_test 1.$nStep.3 {
list [file exists test.db-oal] [file exists test.db-wal]
} {0 1}
do_test 1.$nStep.4 {
list [catch { ota close } msg] $msg
} {0 SQLITE_OK}
}
# Test the outcome of some other client writing the db after the *-oal
# file has been copied to the *-wal path. Once this has happened, any
# other client writing to the db causes OTA to consider its job finished.
#
for {set nStep 8} {$nStep < 20} {incr nStep} {
do_test 1.$nStep.1 {
setup_test
sqlite3ota ota test.db ota.db
for {set i 0} {$i<$nStep} {incr i} {ota step}
ota close
sqlite3 db test.db
execsql { INSERT INTO t1 VALUES(5, 'hello') }
sqlite3ota ota test.db ota.db
ota step
} {SQLITE_DONE}
do_test 1.$nStep.2 {
ota step
} {SQLITE_DONE}
do_test 1.$nStep.3 {
file exists test.db-oal
} {0}
do_test 1.$nStep.4 {
list [catch { ota close } msg] $msg
} {0 SQLITE_DONE}
do_execsql_test 1.$nStep.5 {
SELECT * FROM t1;
} {1 t1 5 hello}
}
finish_test

38
ext/ota/sqlite3ota.c

@ -1303,29 +1303,35 @@ sqlite3ota *sqlite3ota_open(const char *zTarget, const char *zOta){
}
assert( p->rc!=SQLITE_OK || p->eStage!=0 );
if( p->eStage==OTA_STAGE_OAL ){
if( p->rc==SQLITE_OK ){
if( p->rc==SQLITE_OK ){
if( p->eStage==OTA_STAGE_OAL ){
const char *zScript =
"PRAGMA journal_mode=off;"
"PRAGMA pager_ota_mode=1;"
"PRAGMA ota_mode=1;"
"BEGIN IMMEDIATE;"
;
;
p->rc = sqlite3_exec(p->db, zScript, 0, 0, &p->zErrmsg);
/* Point the object iterator at the first object */
if( p->rc==SQLITE_OK ){
p->rc = otaObjIterFirst(p, &p->objiter);
}
if( p->rc==SQLITE_OK ){
otaLoadTransactionState(p, pState);
}
}else if( p->eStage==OTA_STAGE_CKPT ){
p->rc = sqlite3_ckpt_open(
p->db, pState->pCkptState, pState->nCkptState, &p->pCkpt
);
if( p->rc==SQLITE_MISMATCH ){
p->eStage = OTA_STAGE_DONE;
p->rc = SQLITE_DONE;
}
}else if( p->eStage==OTA_STAGE_DONE ){
p->rc = SQLITE_DONE;
}
/* Point the object iterator at the first object */
if( p->rc==SQLITE_OK ){
p->rc = otaObjIterFirst(p, &p->objiter);
}
if( p->rc==SQLITE_OK ){
otaLoadTransactionState(p, pState);
}
}else if( p->rc==SQLITE_OK && p->eStage==OTA_STAGE_CKPT ){
p->rc = sqlite3_ckpt_open(
p->db, pState->pCkptState, pState->nCkptState, &p->pCkpt
);
}
otaFreeState(pState);

15
manifest

@ -1,5 +1,5 @@
C Merge\sversion-3.8.7\schanges\swith\sthis\sbranch.
D 2014-10-20T16:34:03.980
C Add\stests\sfor\sanother\sapplication\swriting\sthe\sdatabase\swhile\san\sota\supdate\sis\songoing.
D 2014-10-21T18:09:58.794
F Makefile.arm-wince-mingw32ce-gcc d6df77f1f48d690bd73162294bbba7f59507c72f
F Makefile.in cf57f673d77606ab0f2d9627ca52a9ba1464146a
F Makefile.linux-gcc 91d710bdc4998cb015f39edf3cb314ec4f4d7e23
@ -122,14 +122,15 @@ F ext/misc/totype.c 4a167594e791abeed95e0a8db028822b5e8fe512
F ext/misc/vfslog.c fe40fab5c077a40477f7e5eba994309ecac6cc95
F ext/misc/vtshim.c babb0dc2bf116029e3e7c9a618b8a1377045303e
F ext/misc/wholenumber.c 784b12543d60702ebdd47da936e278aa03076212
F ext/ota/README.txt cb11e39bfeba952ac8896dab860ada9d54731fb8
F ext/ota/README.txt 78d4a9f78f567d4bf826cf0f02df6254902562ca
F ext/ota/ota.c c11a85af71dccc45976622fe7a51169a481caa91
F ext/ota/ota1.test 0c8e5ef3d059bebe7872477381424836326a8e0a
F ext/ota/ota2.test 4568c2671d19dbde789fb9091d727a2e94880128
F ext/ota/ota3.test 215dd4a8e238567e0f890a5139b6fdf5494ef311
F ext/ota/ota4.test 60f897f329a6782ef2f24862640acf3c52e48077
F ext/ota/ota5.test ad0799daf8923ddebffe75ae8c5504ca90b7fadb
F ext/ota/sqlite3ota.c 1b4e4cdb05e67982865466f4b0aaea8d3648269c
F ext/ota/ota6.test 82f1f757ec9b2ad07d6de4060b8e3ba8e44dfdd3
F ext/ota/sqlite3ota.c 2ae05cfc4e45cf847ba5ec79953b9b55bcb9fc8d
F ext/ota/sqlite3ota.h 7b20abe9247d292429d00f0a5c237ff6e0dc0196
F ext/rtree/README 6315c0d73ebf0ec40dedb5aa0e942bc8b54e3761
F ext/rtree/rtree.c 57bec53e1a677ab74217fe1f20a58c3a47261d6b
@ -1214,7 +1215,7 @@ F tool/vdbe_profile.tcl 67746953071a9f8f2f668b73fe899074e2c6d8c1
F tool/warnings-clang.sh f6aa929dc20ef1f856af04a730772f59283631d4
F tool/warnings.sh 0abfd78ceb09b7f7c27c688c8e3fe93268a13b32
F tool/win/sqlite.vsix deb315d026cc8400325c5863eef847784a219a2f
P 0bf1301aacb3b717b4cc020fbda9fab0bae331c3 19fe4a0a475bd94f491031aea7a183f7c0515cf3
R 348e09d90bfec6d435f7913d476a2e51
P d380a6482a46478ebdf97e08b2fcf78b5d126dde
R 97e1b4279cf76e9ec95460929d6af6ea
U dan
Z d6c8c1f51d897af1aed5284afd8b6a66
Z f79fd33fec0991480e954c839234840a

2
manifest.uuid

@ -1 +1 @@
d380a6482a46478ebdf97e08b2fcf78b5d126dde
2402baa0027ca65e9a5106b8b9c4e10f40d2cbc3