mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,412 Commits 1,487 Branches 166 Tags 416 MiB
5e82c06cf8
Commit Graph

26412 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
drh
9a184daad3 Fix jsonForceRCStr() to also add the NULL terminator. 
FossilOrigin-Name: 134b01f37f8f741d7f7b7eda81384695d1cbe4c39751d87f08832d5c9afdcef2
 2023-07-26 11:00:47 +00:00
stephan
3ba6914121 Reformulate [907dfc4a7aa1] using awk instead of sed for better cross-platform portability. 
FossilOrigin-Name: 82ff7cc6a4b0331677be87bc069da414a56fd531bae402d0f0808b5d2b0d45da
 2023-07-26 07:57:55 +00:00
drh
a757026c5e Merge the latest trunk fixes into the json-opt branch. 
FossilOrigin-Name: ef4e1664d159b98854d9aa580b0bb942f1726f32a190e2ea659c171131ddce9a
 2023-07-26 01:15:34 +00:00
drh
02e683e467 Add the concept of an RCStr string - a "Reference Counted String". Use 
these to keep track of big JSON strings that might need to be used in
multiple places at the same time with different lifetimes.  Add two
different JSON string values to each parse - one with and one without
changes.  The net result is a large reduction in the number of parses
that occur when doing an UPDATE of a indexed JSON field using
json_replace() or similar.

FossilOrigin-Name: a4c1af616e672a0d4d04f2652e645617758231598fb7161b956883512303ae87
 2023-07-26 01:12:08 +00:00
drh
44f53b9647 Extend the enhancement to json_set() and json_replace(). Clean up cruft. 
FossilOrigin-Name: 2dbb22c75e86f2e3ced38ac14b4943570d5c2f86cd5e37e875bf0c863be28836
 2023-07-26 01:05:08 +00:00
drh
0f200bc580 Add the JsonParse.zAlt field to old revised JSON text after a change. 
Demonstrate that this elminates the need for reparsing after a change
by using it in the json_remove() function.  This is an incremental check-in
containing lots of cruft.

FossilOrigin-Name: f930b139d6db0ee799bc90397b225175103c4bf22923d1c9cbcd32509adc1738
 2023-07-26 00:48:45 +00:00
drh
59b8e666f6 Clarify ownership of the various objects involved in parsing JSON. 
FossilOrigin-Name: afe02a398a16d51bd7482b6fbe2fbd15d9ac4fd9cdbc9d2bf81f38b3391fc567
 2023-07-25 20:26:47 +00:00
drh
440e696d5e Incremental improvements to JSON parsing - trying to fold in the RCStr object. 
FossilOrigin-Name: 4cb15d934a85ebc290fe6dd8cd3bd47b159561ca75d72bbffef30b9ea4623b09
 2023-07-25 18:28:03 +00:00
dan
0c39b13e46 Fix a harmless "set-but-not-used" compiler warning in sqlite3session.c. 
FossilOrigin-Name: 54b3c43fdfdaca6b129a5f0ee93c34eb001663775d33c087066650f5e164d1c1
 2023-07-25 17:54:25 +00:00
dan
cbf4a0fa8e Merge trunk changes into this branch. 
FossilOrigin-Name: 719973d7f5a47b110e9919fcb96d21feab1e41356dbb3ec674c1116c17bbb778
 2023-07-25 17:11:29 +00:00
dan
5326953e57 Add extra test for 'deletemerge'. 
FossilOrigin-Name: bc33cff4203cef12518e0f43d380a06d53d67c725fb96cfe6e934b7dc97a7efd
 2023-07-25 16:48:54 +00:00
dan
3f874b58fb Change the name of the fts5 'delete-automerge' option to 'deletemerge'. And add tests for it. 
FossilOrigin-Name: 1079300db2a7d1fbc86a01c215c234a3af64889c5396e6da63ff4f3c7efae4c5
 2023-07-25 15:48:58 +00:00
drh
4bca124845 Fix a minor problem with error reporting in JSON. 
FossilOrigin-Name: c456e4a8999066cd96246327101b3cca78294511a71a2ac07939bb702bfcb5f4
 2023-07-25 15:43:01 +00:00
drh
f02cc9a324 Create the new RCStr class of strings and try to use them for JSON storage. 
FossilOrigin-Name: c1b8725089bb3d006ec69add28f4fcb3f4e79412c7f438b5b1067c2227e77b9c
 2023-07-25 15:08:18 +00:00
dan
039d494d5f Further tests for 'delete-automerge'. 
FossilOrigin-Name: ca26c7a37a7e680be633f43be28f8877bdf9917448ea51c3bedc9b2352a00601
 2023-07-25 13:53:42 +00:00
stephan
322a325703 Reformulate a (sed -i) construct in ext/wasm/GNUmakefile to account for Mac's sed -i being different than GNU's. 
FossilOrigin-Name: 907dfc4a7aa129cdcedeb3ba2d75e1b68a8f22c2545ee1c8cf7d705041644e5c
 2023-07-25 12:26:05 +00:00
drh
b715fe9d80 It is an error to try to insert a BLOB value into JSON. 
FossilOrigin-Name: 28c6e964b1e2257527df02b352f02e135f7c4b764b8b41eda6461f9538f5d042
 2023-07-24 23:27:05 +00:00
drh
3d2a559ecb Clean up some #defines in json. 
FossilOrigin-Name: 327fff501e36f75d4901c520123c5ca45e0e0da1d9cc8fa8fa877ceb68c686d2
 2023-07-24 22:45:59 +00:00
drh
27553579c0 Rework the JSON parse structure to facilitate better caching. Passes all 
tests.

FossilOrigin-Name: ecdcb1ded76e9a0591bf7a2009679f49fc3aa639d3cc12406c6d29243ed8e1c5
 2023-07-24 22:34:26 +00:00
dan
24730de8d1 Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10. 
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
 2023-07-24 19:13:06 +00:00
drh
9125d5ab6c Incremental progress toward improved caching of parsed JSON. 
FossilOrigin-Name: f2c063884685a79d5a787590447c292f51e898a98c9508159c788f505227ba85
 2023-07-24 17:59:25 +00:00
stephan
c76a8862a7 Resolve an ES6 module export construct which is incompatible with node.js, as reported in [forum:b9680fa9ad|forum post b9680fa9ad]. 
FossilOrigin-Name: 80927c3913561dddf75cf73be871d93ae06b16f83e8cc36fc360765014209615
 2023-07-24 15:41:58 +00:00
drh
7d70571142 Merge the latest trunk enhancements into the json-opt branch. 
FossilOrigin-Name: 00bfc4918be49ac74a3e7851c88ab7ec226e6a37853f8ad4c77f758751960456
 2023-07-24 12:37:23 +00:00
stephan
479cfd5af3 When writing the filename header in a opfs-sahpool file, ensure that all remaining bytes in that part of the header are zeroed out to avoid the downstream problems reported in [forum:d50ec48a293988a5|forum post d50ec48a293988a5]. 
FossilOrigin-Name: c1b080e39397c983c13a5e79303223827de7b4946c18a79396851ec1814782f3
 2023-07-23 22:14:41 +00:00
stephan
2ecadd8869 Add the opfs-sahpool sqlite3_vfs implementation to JS, offering an alternative to the other OPFS VFS (with tradeoffs). 
FossilOrigin-Name: d2e602cda44bf35e76167143262b4f91826d25780d0e095e680a31d5dedb2018
 2023-07-22 19:57:42 +00:00
dan
2159292ce0 Integrate contentless delete with auto-merge. 
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
 2023-07-22 19:47:46 +00:00
drh
5d03b1610f Do not read past the end of a text buffer looking for a zero terminator, as 
that space might not be initialized.  If the buffer is owned, just set the
null terminator.  This is a better fix for the OSSFuzz-detected
use-of-initialized-value problem.

FossilOrigin-Name: 931bccb0cc290b8bf3027641e7a7fac30e3244d7dc84aa9e38b24b7e9544ca06
 2023-07-22 16:37:28 +00:00
drh
61a5b6e3bc Fix a harmless use-of-initialized-value warning from OSSFuzz that results 
from recent large string optmizations.

FossilOrigin-Name: 1a6b3dd1c40277a0d0f0bb562ddc4868aadd632fc2d29be1b17bb33fc22c46c8
 2023-07-22 15:21:41 +00:00
drh
b2ee272075 Performance enhancements for queries using very large string values. 
FossilOrigin-Name: 3661b9cd98565ea23b0cafd030668f09885dcbd54eb12b7b3cd85c289aef4c3c
 2023-07-21 22:28:34 +00:00
drh
21f22f9088 Mark an unreachable branch as NEVER(). 
FossilOrigin-Name: 414a4d660c3b505640227066b468db1268fcb1bd3fad931644a769617c6c00cd
 2023-07-21 22:22:13 +00:00
dan
330e36c2c6 Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages. 
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
 2023-07-21 21:10:33 +00:00
dan
4e0c157d02 Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages. 
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
 2023-07-21 19:33:35 +00:00
drh
89f07b1276 Fix harmless compiler warning introduced by the previous check-in. 
FossilOrigin-Name: 06f829e9e57c5aa495f519ad7bb379be611e8e21ef60d36e63d055e45f7a6117
 2023-07-21 19:17:26 +00:00
drh
028acd974f Avoid a potentially large strlen() in sqlite3DecOrHexToI64(). 
FossilOrigin-Name: 5413b02bb629b9cbb76f7e688e94ebcf24276c01436d3feb73ff57c036e1d2aa
 2023-07-21 18:38:59 +00:00
drh
569700a72e Further improvements to large string handling in relation to JSON. 
FossilOrigin-Name: 1e5df0aa3dae5cadbf1d07c718ae2a5212543300b68e49d35e8c96855a7f619c
 2023-07-21 18:09:07 +00:00
drh
eee8687a9f Fix harmless compiler warning about signed/unsigned comparison. 
FossilOrigin-Name: 75cc3c89ee2dcfefa9421ce60bee77e85d2895918e8c5cfd05c434f8932a99b5
 2023-07-21 15:07:56 +00:00
drh
7a2280fe65 Multiple optimizations that try to preserve or infer the zero-terminated 
property of TEXT values.  Avoid unnecessary copying of text values destined
to become function parameters.  All changes help improve performance of
doing UPDATEs on large JSON values that are indexed multiple ways.

FossilOrigin-Name: d0278cdedfa04fb0b61838ab9622be8a2c462f58d5c3ebc4c5f802a727d0974e
 2023-07-21 15:01:53 +00:00
drh
c8f4fec2de More infrastructure changes towards improving JSON cache performance. 
Incremental check-in.

FossilOrigin-Name: 1955e66cfc4614df97b8d68b0e662f309513d62dc8aeec71af5a54e66b79c707
 2023-07-21 11:09:53 +00:00
stephan
bfe6dd0100 Minor internal cleanups in the opfs-sahpool VFS. 
FossilOrigin-Name: 74ad31e2908af8225b7aa527dbcd1877423d58163e365317a78453b31e322ea3
 2023-07-21 10:51:35 +00:00
stephan
3068d6622e Internal cleanups in the async part of the JS library bootstrap phase. 
FossilOrigin-Name: b6d57ab63793241a500ea527c5b3216c54b3ff1972d3adbbf42a9a53bfec0aa1
 2023-07-21 09:10:42 +00:00
stephan
35325a37ce Minor internal cleanups in JS code. 
FossilOrigin-Name: 21a2ca9fc46bf746874579897872e2a45cb07f278abb670dd22b122f7d6a9a6c
 2023-07-20 23:25:32 +00:00
dan
3a51f8c307 Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records. 
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
 2023-07-20 20:29:56 +00:00
dan
d6f5aa824e Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages. 
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
 2023-07-20 20:09:26 +00:00
drh
51d7869096 Experimental framework upon which to build a better JSON parse structure that 
supports cached of modified JSON.  All of these changes are tentative and
subject to change or removal.  Incremental check-in.

FossilOrigin-Name: e384163a4763c10b5838cbf28f9c4d8ea799bbbed6e890e8aa91b2bd1458646d
 2023-07-20 17:45:09 +00:00
dan
0b3791b537 Add tests and fixes for the new code on this branch. 
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
 2023-07-20 16:07:04 +00:00
stephan
0b518b8d65 Filter the OPFS VFSes out of the sqlite3-node.mjs build. Add another level of subdirectory to the sahpool to later enable transparent support of client-provided files under the VFS's root dir. Rework the awkward sahpool-via-oo1 mapping. 
FossilOrigin-Name: 080a4d0aba30d8f3802b49be4a113205f069b3bdea8cebf525d654055642ff62
 2023-07-20 09:06:42 +00:00
drh
f9bfc32b73 Performance enhancements to the JSON parser and code generator. 
FossilOrigin-Name: 0a745897d66e9f36ea6e787fad54f0c413d52c226dc96fad36ccd33a0aa1930e
 2023-07-19 23:02:45 +00:00
drh
93b3c1f398 Revise the new JSON parser performance test to make the test run 25 times 
longer, and thus provide a more repeatable number.

FossilOrigin-Name: bee9e403ae47103938aabb9a7a7e120dfa741b464875965e58788a51fa56a8fe
 2023-07-19 20:23:34 +00:00
drh
5e4d77fa88 Describe JSON parser performance testing procedure in test/json/README.md 
FossilOrigin-Name: c47056e158073ee746c233dd2729cf224ea9e38821b15aa71ebf97607e602281
 2023-07-19 19:23:30 +00:00
dan
d1fbaa071b Fix various issues with code added to this branch. 
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
 2023-07-19 18:47:02 +00:00