Commit Graph

144 Commits

Author SHA1 Message Date
drh
ed0c15b52e Lemon enhancements: (1) Do not allocate space for the 'error' non-terminal
if it is not used. (2) Fix an off-by-one problem so that 'unsigned char' can
be used for symbol numbers if the number of symbols is 256.

FossilOrigin-Name: 3b7801acff91905d0e78e06121ebcf2664f6de5f605699dc8726ec9bcb558eb8
2018-04-16 14:31:34 +00:00
drh
12f6839380 Enhance LEMON to show precendence of symbols and all rules in the report
that is generated in parallel to the parser.

FossilOrigin-Name: 602fbd8149b53d8f0e9a223cc1aec912e7df03fca35071e8d707776ce225371c
2018-04-06 19:12:55 +00:00
drh
4396c61959 In LEMON, fix an off-by-one error that can make the lookahead table one
byte too smal.

FossilOrigin-Name: 93792bc58a2eccc7e07b14307388350bb376db32c5055b79a44e4fa8ff91d58e
2017-12-27 15:21:16 +00:00
drh
0d9de99c5c Add support for measuring and reporting coverage of the parser state machine
using the SQLITE_TESTCTRL_PARSER_COVERAGE test-control.

FossilOrigin-Name: 1253a872dbf48656d4efd588ab61223a5ac550d9b2b932249d6ba585276ba573
2017-12-26 18:04:23 +00:00
drh
3a9d6c7156 Enhance LEMON so that it generates the action table in such a way that no
range check is needed on the lookahead table to verify that the next input
token is valid.  This makes the lookahead table slightly larger (about 120
bytes) but helps the parser to run faster.

FossilOrigin-Name: 7eb0198d0102e97e4b7ad9e359d95985e55e09c510ea4b360265ac8feb9ed814
2017-12-25 04:15:38 +00:00
drh
5c8241b875 In the LEMON-generated parser, rearrange the meanings of integer action codes
so that reduce actions occur last.  This means that the most common case
(reduce actions) can be recognized with a single comparison operation, thus
speeding up the main parser loop, slightly.

FossilOrigin-Name: 7bfe7a360261ac7227840db49487c2f0fe338a2f1b868fcaada1e04a8d2b8f7a
2017-12-24 23:38:10 +00:00
drh
59c435a015 Add the "%token" control to the lemon parser. Not currently used by SQLite.
FossilOrigin-Name: a6e4c5ae8f29bc2e7f2088426341254e9281d19db9dc9a14abc376d56dad4c4b
2017-08-02 03:21:11 +00:00
drh
6be95366c7 In the lemon-generated parser, store the number of symbols on the RHS of each
rule as a negative number and add it to the stack pointer, rather than storing
the value as a positive and subtracting it.  This makes the parser faster.

FossilOrigin-Name: b362f0d8ed34839bf3b29d10ed0853ab94245fba135ccd28e619caa6ee6992d5
2017-06-28 13:47:56 +00:00
drh
bd8fcc130a In the lemon-generated parser, automatically promote SHIFTREDUCE actions on
nonterminal systems to pure REDUCE actions, for a performance enhancement.

FossilOrigin-Name: c46d94a8cde816ca383e771721ebd4900de77950cba700e35a26f70fd5b794a4
2017-06-28 11:56:18 +00:00
drh
69bfa55279 Fix a subtle bug in Lemon discovered and reported on the mailing list
by Kelvin Sherlock, who also suggested the correct fix.

FossilOrigin-Name: 304689f8acb53d68e1afed6e6c4332e78132e3d57071b8f94df0d13515b3b3d8
2017-04-26 04:32:17 +00:00
drh
06f60d8e87 Remove end-of-line whitespace from lemon.c.
FossilOrigin-Name: d78355c85f49e139f1ac1a660563865350f0e442652b7bc50b684398f81cc602
2017-04-14 19:46:12 +00:00
drh
14d8855b8f Fix some left-over K&R-C constructs in lemon.c. No changes to the core.
FossilOrigin-Name: a53799059d4ece246610b6c877ab7088ded3548cbca7149a03deea8ac0e27e68
2017-04-14 19:44:15 +00:00
drh
96ada59cbb Fix harmless compiler warnings in the command-line shell and in Lemon.
FossilOrigin-Name: afcdc4a60e357d171156e0de705bf7ad1b37daab
2016-12-29 19:48:46 +00:00
drh
0f832ddc06 Fix a bug in destructor processing of Lemon. That has no impact on the
SQLite grammar.  The bug was introduced by prior work to optimize the
Lemon-generated parser used by SQLite.

FossilOrigin-Name: f9035b8e2ea331801402bcb62b203ab092949770
2016-08-16 16:46:40 +00:00
drh
701b68879c Improved comments on the lempar.c parser template. Adjust the
YY_SHIFT_USE_DFLT constant in the Lemon-generated parser tables so as
to guarantee that it is always out of range of the yy_lookahead[] table.

FossilOrigin-Name: 83622f3f50030e4d6abafb99d99742928aaf157c
2016-08-10 13:30:43 +00:00
drh
756b41ebd4 Enhance Lemon and the parser template so that it can once again build parsers
that have no unreachable branches.

FossilOrigin-Name: 41fd46e2962ba9a1e1f6867567499d1f6f5b8372
2016-05-24 18:55:08 +00:00
drh
c173ad8011 Lemon enhancement: avoid unnecessary reduce actions that convert one
non-terminal into another but have no side effects.

FossilOrigin-Name: a86e782ad1aa6f5a8b2c54f9dcf0fa61960843f3
2016-05-23 16:15:02 +00:00
drh
711c9810dc Fix comment typos and improve clarity of presention in Lemon. The output
should be identical.

FossilOrigin-Name: b91a5b8297756289d45c8fce0d3399c253517eb0
2016-05-23 14:24:31 +00:00
drh
2e55b04da4 More bug fixes to Lemon identified by Kelvin Sherlock. None of these
have any impact on SQLite.

FossilOrigin-Name: 762bdc55f8878ee2ef65af2165a8e7fdbddf0160
2016-04-30 17:19:30 +00:00
drh
afb8cd977d Lemon bug fix: Do not merge action routines unless their destructors are
also identical.  Problem and suggested fix reported on the mailing list
by Kelvin Sherlock.

FossilOrigin-Name: 4792d6dbba6857f74d27332dcc1468e39c767c71
2016-04-29 11:28:35 +00:00
drh
4ef07708df Enhance Lemon so that it reorders the reduce rules such that rules without
actions occur at the end and so that the first rule is number 0.  This
reduces the size of the jump table on the reduce switch, and helps the parser
to run faster.

FossilOrigin-Name: d5712f21ec758ff096a7b1bb8ed4fc5ec400ca5d
2016-03-16 19:45:54 +00:00
drh
43303de97e More agressive use of /*A-overwrites-X*/ in the parser. Fix an off-by-one
error in parser stack overflow detection.

FossilOrigin-Name: 417e777701bbf4bd67626d4ca3bc2c5d847f6cd0
2016-02-17 12:34:03 +00:00
drh
cf82f0d3d2 Enhance Lemon so that if reduce code contains a comment of the form
"/*A-overwrites-X*/" then a LHS label A is allowed to overwrite the
RHS label X.

FossilOrigin-Name: 5cfe9545d478a2c500083613dd20e14b2ffce645
2016-02-17 04:33:10 +00:00
drh
dabd04c656 Further improvements to the Lemon-generated code for yy_reduce().
FossilOrigin-Name: ef95a7d6490e33a9af4bc7b4b622de7328742ca7
2016-02-17 01:46:19 +00:00
drh
4dd0d3f819 In Lemon, add the ability for the left-most RHS label to be the same as the
LHS label, causing the LHS values to be written directly into the stack.

FossilOrigin-Name: 4bb94c7c4c3cb3ccad72c2451d88684130dde845
2016-02-17 01:18:33 +00:00
drh
c56fac7483 Fix uses of ctype functions (ex: isspace()) on signed characters in test
programs and in some obscure extensions.  No changes to the core.

FossilOrigin-Name: 34eb6911afee09e779318b79baf953f616200128
2015-10-29 13:48:15 +00:00
drh
0c6dfaa34a Remove an unused local variable from Lemon.
FossilOrigin-Name: fe9ffe5eed7d376f3f08c78c1ce5514c886f3479
2015-09-08 21:16:46 +00:00
drh
337cd0def4 Minor tweaks to Lemon.
FossilOrigin-Name: 986677224a8da5e79fbbd90673f1b595da89c5d6
2015-09-07 23:40:42 +00:00
drh
3bd48ab213 For the Lemon-generated parser, add a new action type SHIFTREDUCE and use it
to further compress the parser tables and improve parser performance.

FossilOrigin-Name: 531c3974b3d586c1989cde905b2fb4681239a570
2015-09-07 18:23:37 +00:00
drh
7e698e9d71 In the "parse.out" output file from Lemon, show addition the complete text
of rules on reduce actions.

FossilOrigin-Name: b6ffb7e471e51ff69668154ad2c8790e466c9d37
2015-09-07 14:22:24 +00:00
drh
c75e0166f0 Improved "Parser Statistics" output (the -s option) for the Lemon parser
generator.

FossilOrigin-Name: 809503e4efcdb498d176e8c0794a5ba0882adef2
2015-09-07 02:23:02 +00:00
drh
2547336dad Fix over-length source code lines in Lemon.
FossilOrigin-Name: 1efece95ff8777b89558be59277732ba2a68d5ab
2015-09-04 18:03:45 +00:00
mistachkin
8e18922f12 Fix harmless compiler warnings seen with MSVC 2015.
FossilOrigin-Name: d05becd873a03a366843a34e7f4c732dd8f88c86
2015-04-19 21:43:16 +00:00
drh
1cc0d11a16 Fix harmless compiler warnings in lemon.c
FossilOrigin-Name: bbe7dcda689a25860f2104804f00ba0a720c1372
2015-03-31 15:15:48 +00:00
mistachkin
2318d3387d Work in progress on fixing harmless compiler warnings when using -W4 and MSVC.
FossilOrigin-Name: 856dd245ce037f93d5ae2ddeb4fdf949c1e5c8e9
2015-01-12 18:02:52 +00:00
drh
0325d3962a Enhance the "lemon" executable so that it ignores -f, -W, -O, and -I
command-line options.  This permits most of the same options that are passed
to the compiler to also be harmlessly passed to lemon, and thus
simplifies makefiles.

FossilOrigin-Name: da408d128b4301d9fc7a3a00f219dce7ed48bc60
2015-01-01 19:11:22 +00:00
drh
62a223e5dd Modify the %nonassoc directive in lemon so that it generates a run-time
error rather than a parsing conflict.  This changes is due to a bug report
on the mailing list.  SQLite does not use the %nonassoc directive in its
grammar so this change does not affect SQLite.

FossilOrigin-Name: 1925f3a0a2caa709569df015a8e0d26412f1a9ff
2014-06-09 13:11:40 +00:00
mistachkin
7a429658d7 Fix harmless compiler warning in LEMON.
FossilOrigin-Name: f61a70589ac7e05008a362bd9d5b7bde5d07a758
2014-01-14 10:17:21 +00:00
drh
03e1b1f5ff In LEMON, limit the size of the grammar file to 100MB. This ensures that
the program will never experience integer overflow.  To be doubly sure,
use calloc() instead of malloc() when allocating arrays.

FossilOrigin-Name: 29ba458d849ad8864711cbe59fb10447a947e06a
2014-01-11 12:52:25 +00:00
drh
ecaa9d399c Add the "%token_class" directive to the LEMON parser generator. This opens up
the possibility of simplifying the parser.  Also remove all calls to 
sprintf(), strcpy(), and strcat() from LEMON to avoid compiler warnings
on OpenBSD.  (Aside:  It is this change to avoid harmless compiler warnings
that was the cause of the reason spat of bugs.)

FossilOrigin-Name: 8eb48c04bd0a14031488b3160fde67307eb8b35d
2014-01-11 03:27:37 +00:00
drh
61f92cd045 In LEMON, fix a bug in the text formatter introduced by the previous
commit.  Also add the new "%token_class" directive for defining symbolic
names that stand any one of a collection of tokens.

FossilOrigin-Name: da7890ca6b1d8e511377a469047120220e8c3b2d
2014-01-11 03:06:18 +00:00
drh
898799fa00 Do not use sprintf(), strcpy() or strcat() in the implementation of the
lemon parser generator tool, to avoid compiler warnings in OpenBSD.

FossilOrigin-Name: e43c522dde01e134f1adc94f534d2b3eda74afc2
2014-01-10 23:21:00 +00:00
drh
2b51f2153b Fix a harmless compiler warning in lemon.c.
FossilOrigin-Name: 62959c0ce3a2c486ebd82e6511efad0412b944a0
2013-10-11 23:01:02 +00:00
drh
01f75f2d68 In the lemon parser generator, change all hashes to unsigned to avoid
potential problems with signed integer overflow.

FossilOrigin-Name: 8d399a03de63c15908d63ed69140ee15c6275b8d
2013-10-02 20:46:30 +00:00
drh
f2f105d40e Remove tab characters from source code files. Replace them with spaces.
FossilOrigin-Name: 7edd10a960d5ff121e470b0549b0aa9fb7760022
2012-08-20 15:53:54 +00:00
drh
8ba0d1cbca In Lemon, when comparing the output to the *.h file to see if it has changed,
make sure that the proposed new output and the preexisting output are the
same size before deciding that they are the same.

FossilOrigin-Name: 0c2fb18d25217ada7e75dcab8b342bbc632875d8
2012-06-16 15:26:31 +00:00
drh
d25d692252 Fix a typecast problem in lemon that could cause problems on 64-bit machines.
FossilOrigin-Name: 4a5641cc0aa4c49762f4fe73dab4a6612631c0d2
2012-04-18 09:59:56 +00:00
drh
3cb2f6e60e Cosmetic changes to lemon. No changes to core functionality nor impact on
SQLite.

FossilOrigin-Name: 393fc78a18004c839d889de2a25ec046ad6f13fc
2012-01-09 14:19:05 +00:00
drh
7dd1ac600e Fix a bug in lemon in computation of which non-terminals can generate an
empty string.  This bug and the fix make absolutely no difference for the
grammar used by SQLite, but it can make a difference when lemon is used
in other grammars.

FossilOrigin-Name: ce32775b232da894343f62deefaf19b0ec484636
2012-01-07 15:17:18 +00:00
drh
e0a59cf0e1 Fix a total unimportant file descriptor leak in lemon. This is to silence
warning messages.

FossilOrigin-Name: e95cf2c576dda656c0f31eeec3d98e911b9003a1
2011-08-30 00:58:58 +00:00