mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Ensure that the xColumnText(), xQueryPhrase() and xPhraseFirstColumn() APIs all return SQLITE_RANGE if they are passed a bad column or phrase number.

Browse Source 
FossilOrigin-Name: 1a8a9b1c89519d265869251e8b6d3c5db733f0d3a7dea6c7962811a8f1157dff
This commit is contained in:
dan 2023-12-26 15:52:40 +00:00
parent b2b7490862
commit d82320ac9a
9 changed files with 157 additions and 58 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
ext/fts5/fts5.h
View File

@ -88,8 +88,11 @@ struct Fts5PhraseIter {
** created with the "columnsize=0" option.
**
** xColumnText:
** This function attempts to retrieve the text of column iCol of the
** current document. If successful, (*pz) is set to point to a buffer
** If parameter iCol is less than zero, or greater than or equal to the
** number of columns in the table, SQLITE_RANGE is returned.
**
** Otherwise, this function attempts to retrieve the text of column iCol of
** the current document. If successful, (*pz) is set to point to a buffer
** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
** (not characters) of the buffer and SQLITE_OK is returned. Otherwise,
** if an error occurs, an SQLite error code is returned and the final values
@ -99,8 +102,10 @@ struct Fts5PhraseIter {
** Returns the number of phrases in the current query expression.
**
** xPhraseSize:
** Returns the number of tokens in phrase iPhrase of the query. Phrases
** are numbered starting from zero.
** If parameter iCol is less than zero, or greater than or equal to the
** number of phrases in the current query, as returned by xPhraseCount,
** 0 is returned. Otherwise, this function returns the number of tokens in
** phrase iPhrase of the query. Phrases are numbered starting from zero.
**
** xInstCount:
** Set *pnInst to the total number of occurrences of all phrases within
@ -116,12 +121,13 @@ struct Fts5PhraseIter {
** Query for the details of phrase match iIdx within the current row.
** Phrase matches are numbered starting from zero, so the iIdx argument
** should be greater than or equal to zero and smaller than the value
** output by xInstCount().
** output by xInstCount(). If iIdx is less than zero or greater than
** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned.
**
** Usually, output parameter *piPhrase is set to the phrase number, *piCol
** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol
** to the column in which it occurs and *piOff the token offset of the
** first token of the phrase. Returns SQLITE_OK if successful, or an error
** code (i.e. SQLITE_NOMEM) if an error occurs.
** first token of the phrase. SQLITE_OK is returned if successful, or an
** error code (i.e. SQLITE_NOMEM) if an error occurs.
**
** This API can be quite slow if used with an FTS5 table created with the
** "detail=none" or "detail=column" option.
@ -147,6 +153,10 @@ struct Fts5PhraseIter {
** Invoking Api.xUserData() returns a copy of the pointer passed as
** the third argument to pUserData.
**
** If parameter iPhrase is less than zero, or greater than or equal to
** the number of phrases in the query, as returned by xPhraseCount(),
** this function returns SQLITE_RANGE.
**
** If the callback function returns any value other than SQLITE_OK, the
** query is abandoned and the xQueryPhrase function returns immediately.
** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK.
@ -268,17 +278,25 @@ struct Fts5PhraseIter {
** to a buffer containing the requested token, and *pnToken to the
** size of this buffer in bytes.
**
** If iPhrase or iToken are less than zero, or if iPhrase is greater than
** or equal to the number of phrases in the query as reported by
** xPhraseCount(), or if iToken is equal to or greater than the number of
** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken
are both zeroed.
**
** The output text is not a copy of the query text that specified the
** token. It is the output of the tokenizer module. For tokendata=1
** tables, this includes any embedded 0x00 and trailing data.
**
** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken)
** This is used to access token iToken of phrase hit iIdx within the
** current row. Output variable (*ppToken) is set to point to a buffer
** containing the matching document token, and (*pnToken) to the size
** of that buffer in bytes. This API is not available if the specified
** token matches a prefix query term. In that case both output variables
** are always set to 0.
** current row. If iIdx is less than zero or greater than or equal to the
** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise,
** output variable (*ppToken) is set to point to a buffer containing the
** matching document token, and (*pnToken) to the size of that buffer in
** bytes. This API is not available if the specified token matches a
** prefix query term. In that case both output variables are always set
** to 0.
**
** The output text is not a copy of the document text that was tokenized.
** It is the output of the tokenizer module. For tokendata=1 tables, this

6
ext/fts5/fts5_aux.c
View File

@ -252,8 +252,10 @@ static void fts5HighlightFunction(
ctx.zClose = (const char*)sqlite3_value_text(apVal[2]);
ctx.iRangeEnd = -1;
rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn);
if( ctx.zIn ){
if( rc==SQLITE_RANGE ){
sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC);
rc = SQLITE_OK;
}else if( ctx.zIn ){
if( rc==SQLITE_OK ){
rc = fts5CInstIterInit(pApi, pFts, iCol, &ctx.iter);
}

8
ext/fts5/fts5_expr.c
View File

@ -1890,8 +1890,12 @@ int sqlite3Fts5ExprClonePhrase(
Fts5ExprPhrase *pOrig; /* The phrase extracted from pExpr */
Fts5Expr *pNew = 0; /* Expression to return via *ppNew */
TokenCtx sCtx = {0,0,0}; /* Context object for fts5ParseTokenize */
if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){
rc = SQLITE_RANGE;
}else{
pOrig = pExpr->apExprPhrase[iPhrase];
pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr));
}
if( rc==SQLITE_OK ){
pNew->apExprPhrase = (Fts5ExprPhrase**)sqlite3Fts5MallocZero(&rc,
sizeof(Fts5ExprPhrase*));
@ -1918,6 +1922,7 @@ int sqlite3Fts5ExprClonePhrase(
}
}
if( rc==SQLITE_OK ){
if( pOrig->nTerm ){
int i; /* Used to iterate through phrase terms */
sCtx.pConfig = pExpr->pConfig;
@ -1925,7 +1930,7 @@ int sqlite3Fts5ExprClonePhrase(
int tflags = 0;
Fts5ExprTerm *p;
for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){
rc = fts5ParseTokenize((void*)&sCtx, tflags, p->pTerm,p->nFullTerm,0,0);
rc = fts5ParseTokenize((void*)&sCtx,tflags,p->pTerm,p->nFullTerm,0,0);
tflags = FTS5_TOKEN_COLOCATED;
}
if( rc==SQLITE_OK ){
@ -1938,6 +1943,7 @@ int sqlite3Fts5ExprClonePhrase(
** no token characters at all. (e.g ... MATCH '""'). */
sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase));
}
}
if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){
/* All the allocations succeeded. Put the expression object together. */

16
ext/fts5/fts5_main.c
View File

@ -1911,7 +1911,10 @@ static int fts5ApiColumnText(
){
int rc = SQLITE_OK;
Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab))
Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
if( iCol<0 || iCol>=pTab->pConfig->nCol ){
rc = SQLITE_RANGE;
}else if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab))
|| pCsr->ePlan==FTS5_PLAN_SPECIAL
){
*pz = 0;
@ -1936,8 +1939,9 @@ static int fts5CsrPoslist(
int rc = SQLITE_OK;
int bLive = (pCsr->pSorter==0);
if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){
if( iPhrase<0 || iPhrase>=sqlite3Fts5ExprPhraseCount(pCsr->pExpr) ){
rc = SQLITE_RANGE;
}else if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){
if( pConfig->eDetail!=FTS5_DETAIL_FULL ){
Fts5PoslistPopulator *aPopulator;
int i;
@ -1961,6 +1965,7 @@ static int fts5CsrPoslist(
CsrFlagClear(pCsr, FTS5CSR_REQUIRE_POSLIST);
}
if( rc==SQLITE_OK ){
if( pCsr->pSorter && pConfig->eDetail==FTS5_DETAIL_FULL ){
Fts5Sorter *pSorter = pCsr->pSorter;
int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]);
@ -1969,6 +1974,11 @@ static int fts5CsrPoslist(
}else{
*pn = sqlite3Fts5ExprPoslist(pCsr->pExpr, iPhrase, pa);
}
}else{
*pa = 0;
*pn = 0;
}
return rc;
}

20
ext/fts5/test/fts5_common.tcl
View File

@ -61,6 +61,12 @@ proc fts5_test_collist {cmd} {
set res
}
proc fts5_collist {cmd iPhrase} {
set res [list]
$cmd xPhraseColumnForeach $iPhrase c { lappend res $c }
set res
}
proc fts5_test_columnsize {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@ -69,6 +75,10 @@ proc fts5_test_columnsize {cmd} {
set res
}
proc fts5_columntext {cmd iCol} {
$cmd xColumnText $iCol
}
proc fts5_test_columntext {cmd} {
set res [list]
for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@ -125,6 +135,13 @@ proc fts5_test_queryphrase {cmd} {
set res
}
proc fts5_queryphrase {cmd iPhrase} {
set cnt [list]
for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 }
$cmd xQueryPhrase $iPhrase [list test_queryphrase_cb cnt]
set cnt
}
proc fts5_test_phrasecount {cmd} {
$cmd xPhraseCount
}
@ -154,6 +171,9 @@ proc fts5_aux_test_functions {db} {
fts5_test_queryphrase
fts5_test_phrasecount
fts5_columntext
fts5_queryphrase
fts5_collist
} {
sqlite3_fts5_create_function $db $f $f
}

43
ext/fts5/test/fts5aux.test
View File

@ -334,4 +334,47 @@ do_execsql_test 11.2 {
SELECT fts5_hitcount(x1) FROM x1('one') LIMIT 1;
} {5}
#-------------------------------------------------------------------------
# Test that xColumnText returns SQLITE_RANGE when it should.
#
reset_db
fts5_aux_test_functions db
do_execsql_test 12.0 {
CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
INSERT INTO t1 VALUES('one', 'two', 'three');
INSERT INTO t1 VALUES('one', 'one', 'one');
INSERT INTO t1 VALUES('two', 'two', 'two');
INSERT INTO t1 VALUES('three', 'three', 'three');
}
do_catchsql_test 12.1.1 {
SELECT fts5_columntext(t1, -1) FROM t1('two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.1.2 {
SELECT fts5_columntext(t1, 3) FROM t1('two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.1.2 {
SELECT fts5_columntext(t1, 1) FROM t1('one AND two');
} {0 two}
do_catchsql_test 12.2.1 {
SELECT fts5_queryphrase(t1, -1) FROM t1('one AND two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.2.2 {
SELECT fts5_queryphrase(t1, 2) FROM t1('one AND two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.2.3 {
SELECT fts5_queryphrase(t1, 1) FROM t1('one AND two');
} {0 {{1 2 1}}}
do_catchsql_test 12.3.1 {
SELECT fts5_collist(t1, -1) FROM t1('one AND two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.3.2 {
SELECT fts5_collist(t1, 2) FROM t1('one AND two');
} {1 SQLITE_RANGE}
do_catchsql_test 12.3.3 {
SELECT fts5_collist(t1, 1) FROM t1('one AND two');
} {0 1}
finish_test

2
ext/fts5/test/fts5synonym2.test
View File

@ -42,7 +42,7 @@ proc fts5_test_bothlist {cmd} {
}
sqlite3_fts5_create_function db fts5_test_bothlist fts5_test_bothlist
proc fts5_rowid {cmd} { expr [$cmd xColumnText -1] }
proc fts5_rowid {cmd} { expr [$cmd xRowid] }
sqlite3_fts5_create_function db fts5_rowid fts5_rowid
do_execsql_test 1.$tok.0.1 "

26
manifest
View File

@ -1,5 +1,5 @@
C Improved\shandling\sof\smalformed\sunicode\swithin\sJSON\sstrings.
D 2023-12-26T13:20:57.593
C Ensure\sthat\sthe\sxColumnText(),\sxQueryPhrase()\sand\sxPhraseFirstColumn()\sAPIs\sall\sreturn\sSQLITE_RANGE\sif\sthey\sare\spassed\sa\sbad\scolumn\sor\sphrase\snumber.
D 2023-12-26T15:52:40.355
F .fossil-settings/empty-dirs dbb81e8fc0401ac46a1491ab34a7f2c7c0452f2f06b54ebb845d024ca8283ef1
F .fossil-settings/ignore-glob 35175cdfcf539b2318cb04a9901442804be81cd677d8b889fcc9149c21f239ea
F LICENSE.md df5091916dbb40e6e9686186587125e1b2ff51f022cc334e886c19a0e9982724
@ -90,15 +90,15 @@ F ext/fts3/unicode/UnicodeData.txt cd07314edb62d49fde34debdaf92fa2aa69011e7
F ext/fts3/unicode/mkunicode.tcl d5aebf022fa4577ee8cdf27468f0d847879993959101f6dbd6348ef0cfc324a7
F ext/fts3/unicode/parseunicode.tcl a981bd6466d12dd17967515801c3ff23f74a281be1a03cf1e6f52a6959fc77eb
F ext/fts5/extract_api_docs.tcl bc3a0ca78be7d3df08e7602c00ca48021ebae40682d75eb001bfdf6e54ffb44e
F ext/fts5/fts5.h 9d7867cc63631296462c90bafe57c4881b56e480fa510ffaee6a77049258c714
F ext/fts5/fts5.h ecba24fed7b359b3a53016bb07e411b3b4c9cdf163aa141006536423a63b611e
F ext/fts5/fts5Int.h defa43c0932265138ee910ca416e6baccf8b774e0f3d610e74be1ab2880e9834
F ext/fts5/fts5_aux.c ee770eec0af8646db9e18fc01a0dad7345b5f5e8cbba236704cfae2d777022ad
F ext/fts5/fts5_aux.c 4584e88878e54828bf7d4d0d83deedd232ec60628b7731be02bad6adb62304b1
F ext/fts5/fts5_buffer.c 0eec58bff585f1a44ea9147eae5da2447292080ea435957f7488c70673cb6f09
F ext/fts5/fts5_config.c 8072a207034b51ae9b7694121d1b5715c794e94b275e088f70ae532378ca5cdf
F ext/fts5/fts5_expr.c 248ecadbacdbeb85c433f907e57bb91224678c829e57ecf098e3543b5df8c3f9
F ext/fts5/fts5_expr.c e5fb9dd9e31e9e6ae9604bdb0d183ecec720964f3b974fc37c43ce73d8833d6d
F ext/fts5/fts5_hash.c adda4272be401566a6e0ba1acbe70ee5cb97fce944bc2e04dc707152a0ec91b1
F ext/fts5/fts5_index.c bb1965c3965f6fe5f64160bf1c0694a9684a790a783f293a76da1d38d319b258
F ext/fts5/fts5_main.c b908696c52410e8383019ac0657c8a5cd0c8f60e78edc169e9b3c4b93f24c933
F ext/fts5/fts5_main.c 93e5bc8676dbaaec365576d26cdafc5c21f0052f8f68b86ee4de22237c2e2143
F ext/fts5/fts5_storage.c f9e31b0d155e9b2c92d5d3a09ad7a56b937fbf1c7f962e10f4ca6281349f3934
F ext/fts5/fts5_tcl.c cf0fd0dbe64ec272491b749e0d594f563cda03336aeb60900129e6d18b0aefb8
F ext/fts5/fts5_test_mi.c 08c11ec968148d4cb4119d96d819f8c1f329812c568bac3684f5464be177d3ee
@ -109,7 +109,7 @@ F ext/fts5/fts5_varint.c e64d2113f6e1bfee0032972cffc1207b77af63319746951bf1d0988
F ext/fts5/fts5_vocab.c 209e0c151e108d5f3621fa24b91e9b02f3750ee6c3f9ccec312df39481b68a09
F ext/fts5/fts5parse.y eb526940f892ade5693f22ffd6c4f2702543a9059942772526eac1fde256bb05
F ext/fts5/mkportersteps.tcl 5acf962d2e0074f701620bb5308155fa1e4a63ba
F ext/fts5/test/fts5_common.tcl 8b1848ac2baad10e444e4183034a52050b52d20b3796d9d30e78f01ab0d05583
F ext/fts5/test/fts5_common.tcl 3378732aae2a7d9a4b9b5c40bde678d4259ca16bd490883325aecc4747bcb384
F ext/fts5/test/fts5aa.test 4db81519863244a3cab35795fe65ab6b592e7970c7409eba098b23ebbfc08d95
F ext/fts5/test/fts5ab.test bd932720c748383277456b81f91bc00453de2174f9762cd05f95d0495dc50390
F ext/fts5/test/fts5ac.test a7aa7e1fefc6e1918aa4d3111d5c44a09177168e962c5fd2cca9620de8a7ed6d
@ -124,7 +124,7 @@ F ext/fts5/test/fts5ak.test f459a64c9d38698af72a7c657ab6349bca96150241dd69fcce75
F ext/fts5/test/fts5al.test 00c4c1c6a1366b73aa48ce2068c634520867c3cf7f5d1676ebbb775ee1f35734
F ext/fts5/test/fts5alter.test 5565f7e4605512b69171ac18ca84398603f9f6456dbe377beeca97e83cc242cd
F ext/fts5/test/fts5auto.test 78989e6527ce69c9eddbef7392fea5c10b0010cd2b2ae68eec7bc869c471e691
F ext/fts5/test/fts5aux.test 3f194345fcd581f49f7fbb2e5495400efcc7d2835b77816328d8283c942f41b8
F ext/fts5/test/fts5aux.test ed3596469f85a6cff5f6060e0cd9e3f9602051d8db2b497f5d12c85d39f20a62
F ext/fts5/test/fts5auxdata.test eacc97ff04892f1a5f3d4df5a73f8bcbc3955ea1d12c9f24137eb1fc079e7611
F ext/fts5/test/fts5bigid.test 2860854c2561a57594192b00c33a29f91cb85e25f3d6c03b5c2b8f62708f39dd
F ext/fts5/test/fts5bigpl.test 6466c89b38439f0aba26ac09e232a6b963f29b1cbe1304f6a664fe1e7a8f5fd3
@ -222,7 +222,7 @@ F ext/fts5/test/fts5simple.test a298670508c1458b88ce6030440f26a30673931884eb5f40
F ext/fts5/test/fts5simple2.test 8dd2389ee75e21a1429fe87e5f8c7d9a97ad1470304a8a2d3ba4b8c3c345fecd
F ext/fts5/test/fts5simple3.test d5c74a9d3ca71bd5dd5cacb7c55b86ea12cdddfc8b1910e3de2995206898380f
F ext/fts5/test/fts5synonym.test 1651815b8008de170e8e600dcacc17521d765482ea8f074ae82cfa870d8bb7fb
F ext/fts5/test/fts5synonym2.test 8f891fc49cc1e8daed727051e77e1f42849c784a6a54bef82564761b2cb3e016
F ext/fts5/test/fts5synonym2.test e2f6ff68c4fbe12a866a3a87510f553d9dac99bcb74c10b56487c4c0a562fcf5
F ext/fts5/test/fts5tok1.test 1f7817499f5971450d8c4a652114b3d833393c8134e32422d0af27884ffe9cef
F ext/fts5/test/fts5tok2.test dcacb32d4a2a3f0dd3215d4a3987f78ae4be21a2
F ext/fts5/test/fts5tokenizer.test ac3c9112b263a639fb0508ae73a3ee886bf4866d2153771a8e8a20c721305a43
@ -2156,8 +2156,8 @@ F vsixtest/vsixtest.tcl 6a9a6ab600c25a91a7acc6293828957a386a8a93
F vsixtest/vsixtest.vcxproj.data 2ed517e100c66dc455b492e1a33350c1b20fbcdc
F vsixtest/vsixtest.vcxproj.filters 37e51ffedcdb064aad6ff33b6148725226cd608e
F vsixtest/vsixtest_TemporaryKey.pfx e5b1b036facdb453873e7084e1cae9102ccc67a0
P b9daf37e57cde12c4de271a2b1995e8e91b6411f8c2e8882e536241929609b3a
R 8e5515c216d097520f23a07a88eb7f34
U drh
Z acb4ee2fdca29b1d699d6a53a9b5165e
P e252bdf5f5de26ba8e2bcc6b0ad94121ed6fc4d86c02fe4a2a058ada93747beb
R a8a4e2c094005bc85798feb38e88e2ad
U dan
Z 04de1187c12166f6095f68d3113b177a
# Remove this line to create a well-formed Fossil manifest.

2
manifest.uuid
View File

@ -1 +1 @@
e252bdf5f5de26ba8e2bcc6b0ad94121ed6fc4d86c02fe4a2a058ada93747beb
1a8a9b1c89519d265869251e8b6d3c5db733f0d3a7dea6c7962811a8f1157dff