Commit Graph

17570 Commits

Author SHA1 Message Date
rillig
ac07af747c make: improve documentation of ApplyModifiersState
No functional change.
2021-03-15 19:15:04 +00:00
rillig
a05498199d make: document an example for a 'chain of modifiers'
No functional change.
2021-03-15 19:02:57 +00:00
rillig
507bcc82b8 make: fix double varname expansion in the variable modifier '::='
This is an edge case that doesn't occur in practice since pretty much
nobody dares to use variable names that contain an actual '$' in their
name.  This is not about the fairly common VAR.${param} (as written in
the makefile), but instead about the variable whose name is literally
'VAR.${param}'.

The test demonstrates that after the fix, the variable name is taken
exactly as-is for the simple assignment modifier '::='.  There are no
such tests for the modifiers '::+=', '::!=' and '::?=', but that's ok.
The code in ApplyModifier_Assign would look assymetrical and suspicious
enough if one of these modifiers would expand its variable name and the
others wouldn't.
2021-03-15 18:56:37 +00:00
rillig
7eea180375 tests/make: demonstrate that the modifier '::=' expands the varname 2021-03-15 18:46:05 +00:00
rillig
87c62a5243 tests/make: convert varmod-loop to parse-time
In case of unexpected failures, this provides the line number of the
'.error' directive.
2021-03-15 17:54:49 +00:00
rillig
8d27738a97 tests/make: add position marker in test 'varmod-loop' 2021-03-15 17:11:08 +00:00
rillig
e9a9d1718c make: clean up documentation of ApplyModifiersState
No functional change.
2021-03-15 16:51:14 +00:00
rillig
c066baf306 make: fix documentation of Lst_MoveAll
In CLEANUP mode, was originally meant to track memory allocations but is
useful during debugging as well, initialize the list.  There is no
distinct constant representing an invalid pointer, otherwise that would
have been an even better choice.
2021-03-15 16:45:30 +00:00
rillig
c175751478 make: indent inline functions for lists
No functional change.
2021-03-15 16:06:05 +00:00
rillig
0983b9b566 make: clean up header for runtime type information for enums
An enum with 32 bits would lead to signed integer overflow anyway, so
that definition is not worth keeping even if it works on typical
2-complement platforms.

The definitions for 2, 4 and 8 enum have been unused for several months
now.

No functional change.
2021-03-15 16:00:05 +00:00
rillig
4a38bf21f4 make: change debug log for variable evaluation flags to lowercase
This makes them easier distinguishable from variable names since the
latter are usually uppercase.

No functional change outside debug mode.
2021-03-15 15:39:13 +00:00
rillig
2f8026b0e5 make: replace enum bit-field with struct bit-field for VarEvalFlags
This makes the code easier to read, especially in var.c.  It also makes
debugging sessions easier since some debuggers don't show enum
bit-fields symbolically as soon as more than one bit is set.

The code outside var.c is basically unchanged, except that instead of
passing the individual flags, there are 4 predefined evaluation modes.
These suffice for all practical use cases.  Only in the implementation
deep inside var.c, the value of the flags keepDollar and keepUndef
differs.

There is no way of passing the struct to EnumFlags_ToString, which means
the ToString function has to be spelled out explicitly.  This allows for
fine-tuning the representation in the debug log, to reduce the amount of
uppercae letters.

No functional change.
2021-03-15 12:15:03 +00:00
rillig
4fd3cf6eed make: rename VARE_NONE to VARE_PARSE_ONLY
The name 'NONE' described the bit pattern, which was not useful to
understand its meaning.  Omitting VARE_WANTRES only parses the
expression, without evaluating any part of it.

No functional change, not even in debug mode since Enum_FlagsToString
always returns "none" for all-bits-unset.
2021-03-15 11:41:07 +00:00
rillig
8699d6553b tests/make: document today's bug fixes in the test 2021-03-14 20:41:39 +00:00
rillig
3042e42091 make: fix documentation of VarFreeEnv
No functional change.
2021-03-14 20:23:29 +00:00
rillig
259019554f make: skip variable lookup for '::=' modifiers in parse-only mode
This is just to keep the code consistent among the various variable
modifiers.  The performance gain is negligible.

The actual assignment to the variable had already been skipped
previously.

No functional change.
2021-03-14 20:18:33 +00:00
rillig
96491d26ea make: separate parsing and evaluating for modifier '::='
No functional change.
2021-03-14 20:12:16 +00:00
rillig
d4faf73862 make: in parse-only mode, do not update the expression value in ':sh'
No functional change outside debug mode.  The other variable modifiers
behave in the same way.
2021-03-14 20:09:26 +00:00
rillig
96faeb06ee make: do not evaluate the ':O' modifiers in parse-only mode
No functional change in practical usage.  Theoretically this change can
be observed by looking at the generated random numbers for the ':Ox'
modifier, but the quality or exact sequence of these random numbers is
not guaranteed anyway.
2021-03-14 20:03:56 +00:00
rillig
f7a4d3a3e0 make: separate parsing and evaluating in the ':O' modifiers
No functional change.
2021-03-14 20:00:48 +00:00
rillig
812ef981d6 make: do not evaluate modifier ':[...]' in parse-only mode
In parse-only mode, variable expressions in the argument to that
modifier are not resolved.  This led to the error message about the 'Bad
modifier' in var-eval-short.mk.
2021-03-14 19:29:37 +00:00
rillig
ef2d559187 make: do not evaluate modifiers ':M' and ':N' in parse-only mode
No functional change outside debug mode (-dv for ModifyWord_Match).
2021-03-14 19:25:05 +00:00
rillig
aaaf8460b3 make: do not return unevaluated 'else' part from the ':?' modifier
No functional change outside debug mode.
2021-03-14 19:21:28 +00:00
rillig
91f10d43f6 tests/make: add test for the ':?' modifier in parse-only mode
The debug output for this scenario will change a bit in an upcoming
commit, but that will not affect anything outside the debug log.
2021-03-14 19:16:41 +00:00
rillig
c8091d0b16 make: do not evaluate ':gmtime' and ':localtime' in parse-only mode
No functional change.
2021-03-14 18:30:24 +00:00
rillig
99e5bcf513 make: don't evaluate several simple modifiers in parse-only mode
This affects the modifiers ':E', ':H', ':P', ':Q', ':R', ':T', ':hash',
':q', ':range', ':tl', ':ts', ':tu', and ':u'.  All these modifiers are
side-effect free.

Skipping the evaluation for these modifiers is purely for code
consistency and performance.

No functional change.
2021-03-14 18:23:44 +00:00
rillig
a1837566a4 make: skip strdup when parsing an irrelevant ':L' modifier
No functional change.
2021-03-14 18:10:57 +00:00
rillig
ae0a762a50 make: skip memcpy when parsing but not evaluating ':D' and ':U'
No functional change, just a tiny bit of performance improvement,
probably not even measurable.  Having the code nevertheless serves as a
copy-and-paste template for implementing other modifiers that might
perform more costly tasks.
2021-03-14 18:08:25 +00:00
rillig
a1747f17a6 make: only evaluate the ':@' modifier if the result is actually used
The test 'var-eval-short' had produced the output 'unexpected' before,
on stderr.  It had been generated by '${:Uword:@${FAIL}@expr@}' by
combining the following obscure "features" of make:

1.  the ':@' modifier loops over the words of the variable.  This
    modifier is not really obscure, it still takes some time to get used
    to it.

2.  the ':@' modifier allows a '$' sign in the variable name, which is
    useless in practice.

3.  the ':@' modifier creates a temporary loop variable in the global
    namespace.  Luckily there are only few collisions with other
    variable names since their naming conventions differ.

4.  after looping over the words of the expression, the temporary global
    loop variable is deleted, and at that point the '$' is expanded,
    being interpreted as the start of a variable expression.

5.  The ':@' modifier deleted the global variable even when it was
    called in parse-only mode (without VARE_WANTRES).

When the modifier ':@' was initially added to make in var.c 1.40 from
2000-04-29, Var_Delete didn't expand the variable name.  That feature
was added in var.c 1.174 from 2013-05-18, probably without thinking of
this very edge-casey combination of features.

This commit fixes item 5 from the above list.  The other obscurities
remain for now.
2021-03-14 18:02:44 +00:00
rillig
9ac0661025 make: eliminate common subexpression in ApplyModifier_Remember
No functional change.
2021-03-14 17:38:24 +00:00
rillig
71128ca3aa make: merge duplicate code in ApplyModifier_Remember
This way, parsing and evaluating the modifier is only written once in
the code.  The downside is that the variable name is allocated even if
VARE_WANTRES is not set, but since this modifier is so obscure and
seldom used this doesn't matter in practice.
2021-03-14 17:34:50 +00:00
rillig
0fc517c113 make: do not expand the variable name in the ':_' modifier
This edge case had been so obscure that even discovering this takes
quite some time and requires reading the source code of make.

The manual page doesn't document whether the variable name is expanded
or not, it doesn't even give an example.  When this obscure modifier was
initially added in var.c 1.210 from 2017-01-30, Var_Set always expanded
the variable name once, and there was no way around it.  Therefore this
expansion has probably been unintentional.
2021-03-14 17:27:27 +00:00
rillig
a442584d73 tests/make: demonstrate unintended edge case for the ':_' modifier 2021-03-14 17:14:15 +00:00
rillig
d905f14720 tests/make: convert test for ':_' modifier to parse time 2021-03-14 17:07:11 +00:00
rillig
64a393d8af make: only evaluate the ':_' modifier if the expression is needed
See var-eval-short.mk:46 for the test demonstrating this change.
Previously, the expression ${:Uword:_=VAR} was evaluated including all
its side effects even though it was in an irrelevant branch of the
condition.
2021-03-14 16:43:30 +00:00
rillig
c361e1ef2f make: return failure in TryParseIntBase0 for empty string
No functional change since the only caller of TryParseIntBase0 already
handles all possible parse errors.  Without this check, the code just
looked wrong though.
2021-03-14 16:03:04 +00:00
rillig
f3e066bf1b make: add test for edge case in modifier ':[...]'
TryParseIntBase0 wrongly returns successful for a string that does not
start with a number at all.  Its only caller, ApplyModifier_Words,
already handles all error cases properly.

No functional change.
2021-03-14 16:00:07 +00:00
rillig
c9c8cb6d43 make: separate parsing from evaluating for several modifiers
This aligns the implementation of these modifiers with the requirements
in the long comment starting with 'The ApplyModifier functions'.

No functional change.
2021-03-14 15:43:31 +00:00
rillig
54f9178de3 make: reduce indentation in ApplyModifier_SunShell
No functional change.
2021-03-14 15:24:37 +00:00
rillig
5367d2f7c6 make: separate parsing and evaluating for modifier ':u'
No functional change.
2021-03-14 15:22:21 +00:00
rillig
a0706d9698 make: separate parsing and evaluating for modifiers ':Q' and ':q'
No functional change.
2021-03-14 15:19:15 +00:00
rillig
d74102acf0 make: extract duplicate code for parsing ':S' and ':C'
No functional change.
2021-03-14 15:15:28 +00:00
rillig
3e9e13f502 make: extract duplicate code for detecting a modifier's end
No functional change.
2021-03-14 15:06:19 +00:00
rillig
115656881d make: document how error handling should be done correctly
Right now, when a variable expression cannot be parsed, the result of
calling Var_Subst is a string containing garbage, and no error is
reported.  In addition, there are some silent errors that are not
reported at all.  This combination makes it difficult to change the
error handling without introducing subtle breakage in some edge cases.

An example for garbage output is in varmod-subst-regex.mk, in target
mod-regex-compile-error.

No functional change.
2021-03-14 15:04:13 +00:00
rillig
63e8f1e660 tests/make: add test for short-circuit evaluation of modifiers 2021-03-14 11:49:37 +00:00
rillig
58e152bdc2 make: sort modifiers in ApplyModifier alphabetically
No functional change.
2021-03-14 11:15:37 +00:00
rillig
8902b05acf make: fix wrong expression evaluation in -dL mode
The modifier ':C' now only compiles the regular expression if the result
of the expression is actually needed.

Several other modifiers have the same bug of evaluating the expression
in cases where this is not needed.  It just doesn't show up because they
don't have any noticeable side effects, other than wasting CPU time.
This affects irrelevant conditions as well.
2021-03-14 10:57:12 +00:00
rillig
0ed436993c make: demonstrate wrong expression evaluation in -dL mode 2021-03-14 10:45:51 +00:00
rillig
de1cec24a0 indent: clean up check_size_comment
The additional parameter last_bl_ptr was only necessary because the last
blank was stored as a pointer into the buffer.  By storing the index in
the buffer instead, it doesn't need to be updated all the time.

No functional change.
2021-03-14 05:26:42 +00:00
rillig
a8e5d6abb4 indent: remove trailing whitespace 2021-03-14 04:52:10 +00:00
rillig
f21f69f749 indent: clean up target column computation in process_comment
No functional change.
2021-03-14 04:42:17 +00:00
rillig
3cc06592f5 indent: make compute_code_indent more readable
The '?:' operator computing the factor was too hard to read.  When
quickly scanning the code, the 1 in the expression looked too much like
it would be added to the indentation, which would turn the indentation
length into a column number, and that again would smell like an
off-by-one error.

No functional change.
2021-03-14 01:44:37 +00:00
rillig
1e4c413bac indent: fix off-by-one error in comment wrapping
The manual page says that the default maximum length of a comment line
is 78.  The test 'comments.0' wrongly assumed that this 78 would refer
to the maximum _column_ allowed, which is off by one.

Fix the wording in the test 'comments.0' and remove the (now satisfied)
expectation comments in the test 'token-comment.0'.

Several other tests just happened to hit that limit, fix these as well.
2021-03-14 01:34:13 +00:00
rillig
bd908d33db indent: give indent a try at formatting its own code
Formatting indent.h required the following manual corrections
afterwards:

The first tab in the comment in line 1 was replaced with a space but
shouldn't be.

The spacing around the '...' in function prototypes was completely
wrong.  It looked like 'const char *,...)__printflike', without any
spaces.

The '*' of the return type 'const char *' was tied to the function name,
even though this declaration was only for a single function.  In such a
case, it's more appropriate to line up the function names.

The function-like macros were not indented to -di.  This is something
that I would not expect from indent, so it's ok to do that manually.
2021-03-14 00:33:25 +00:00
rillig
744982a9b6 indent: fix lint warnings
No functional change.
2021-03-14 00:22:16 +00:00
rillig
1eb04b8cb5 indent: remove disabled duplicate RCS ID from header
By convention, headers don't record their RCS ID.
2021-03-13 23:42:23 +00:00
rillig
c960730f11 indent: fix documentation of parser_state.paren_indents
The column position is not the same as the indentation (off-by-one).
2021-03-13 23:36:10 +00:00
rillig
fea66a5127 indent: add debug logging for switching the input buffer
No functional change outside debug mode.
2021-03-13 18:46:39 +00:00
rillig
eb340c3e4a indent: align comments in indent's own code
No functional change.
2021-03-13 18:24:56 +00:00
rillig
56c4653e3f indent: remove the '+ 1' from right margin calculation in comment
No functional change.
2021-03-13 18:11:31 +00:00
christos
cf29492e59 Handle \t too (RVP) 2021-03-13 15:46:54 +00:00
rillig
7e11a5f382 indent: rename local variable in dump_line
This clarifies that the variable names a column, not an indentation.
2021-03-13 13:55:42 +00:00
rillig
7301e2c2d1 indent: in dump_line, reduce scope of local variable
This allows the variable 'target' in the lower half of the function to
get a more specific name.

No functional change.
2021-03-13 13:54:01 +00:00
rillig
66af9142ab indent: distinguish between 'column' and 'indentation'
column == 1 + indentation.

In addition, indentation is a relative distance while column is an
absolute position.  Therefore, don't confuse these two concepts, to
prevent off-by-one errors.

No functional change.
2021-03-13 13:51:08 +00:00
rillig
7ad6720446 indent: rename pr_comment to process_comment, clean up documentation
No functional change.
2021-03-13 13:25:23 +00:00
rillig
93e110b15f indent: fix handling of '/*' in string literal in preprocessing line
Previously, the '/*' in the string literal had been interpreted as the
beginning of a comment, which was wrong.  Because of that, the variable
declaration in the following line was still interpreted as part of the
comment.  The comment even continued until the end of the file.

Due to indent's forgiving nature, it neither complained nor even
mentioned that anything had gone wrong.  The decision of rather
producing wrong output than failing early is a dangerous one.

At least, there should have been an error message that at the end of the
file, the parser was still in a a comment, expecting the closing '*/'.
2021-03-13 13:14:14 +00:00
rillig
102d371a8a indent: split 'main_loop' into several functions
No functional change.
2021-03-13 12:52:24 +00:00
rillig
b97b269d52 indent: split 'main' into manageable parts
Since several years (maybe even decades) compilers know how to inline
static functions that are only used once.  Therefore there is no need to
have overly long functions anymore, especially not 'main', which is only
called a single time and thus does not add any noticeable performance
degradation.

No functional change.
2021-03-13 11:47:22 +00:00
rillig
acec5beac9 indent: remove redundant parentheses
No functional change.
2021-03-13 11:27:01 +00:00
rillig
0a99ae80ca indent: fix confusing variable names
The word 'col' should only be used for the 1-based column number.  This
name is completely inappropriate for a line length since that provokes
off-by-one errors.  The name 'cols' would be acceptable although
confusing since it sounds so similar to 'col'.

Therefore, rename variables that are related to the maximum line length
to 'line_length' since that makes for obvious code and nicely relates to
the description of the option in the manual page.

No functional change.
2021-03-13 11:19:43 +00:00
rillig
2c2459a1fa indent: document undefined behavior in processing of comments
No functional change.
2021-03-13 10:47:59 +00:00
rillig
f3b63c94c8 indent: inline calls to count_spaces and count_spaces_until
These two functions operated on column numbers instead of indentation,
which required adjustments of '+ 1' and '- 1'.  Their names were
completely wrong since these functions did not count anything, instead
they computed the column.

No functional change.
2021-03-13 10:32:25 +00:00
rillig
526591ce10 indent: replace column computation with indentation computation
No functional change.
2021-03-13 10:20:54 +00:00
rillig
0c51d9451c indent: replace compute_code_column with compute_code_indent
The goal is to only ever be concerned about the _indentation_ of a
token, never the _column_ it appears in.  Having only one of these
avoids off-by-one errors.

No functional change.
2021-03-13 10:06:47 +00:00
rillig
5888ddac66 indent: replace compute_label_column with compute_label_indent
Using the invariant 'column == 1 + indent'.  This removes several overly
complicated '+ 1' from the code that are not needed conceptually.

No functional change.
2021-03-13 09:54:11 +00:00
rillig
4f1ab5eff9 indent: manually fix indentation in indent's own source code 2021-03-13 09:48:04 +00:00
rillig
6f2286deb6 indent: add debug logging for actually writing to the output file
Together with the results of the tokenizer and the 4 buffers for token,
label, code and comment, the debug log now provides a good high-level
view on how the indentation happens and where to look for the many
remaining bugs.
2021-03-13 09:21:57 +00:00
rillig
6892e0dc50 indent: remove strange debugging code that went in the output file
Whenever the code to be output contained the magic byte 0x80, instead of
writing this byte, indent wrote the column number at the beginning of
the code snippet, times 7.  Especially the 'times 7' does not make any
sense at all.

In ISO-8859-1, this character position is not assigned.  In Microsoft
Codepage 1252 it is the Euro sign.  In UTF-8 (which was probably not on
the author's list when the code was originally written) it occurs as the
middle byte for code points like U+2026 (horizontal ellipsis) from the
block General Punctuation.

Remove this strange code, thereby fixing indent for UTF-8 code.  The
code had been there since at least 1993-04-09, when it was first
imported to NetBSD.
2021-03-13 09:06:12 +00:00
rillig
689a1f7922 indent: replace pad_output with output_indent
Calculating the indentation is simpler than calculating the column,
since that saves the constant addition and subtraction of the 1.

No functional change.
2021-03-13 00:26:56 +00:00
rillig
47a823baba indent: clean up verbose documentation comments from the 1970s
Since C90, there is no need to repeat the type of the function
parameters.

In the whole code of indent, there is a lot of confusion between the
concepts of a 'column' (which is a position on the screen, counting
starts at 1) and 'indentation' (which is a length, not a position).  To
avoid this confusion, the code will be rewritten anyway very soon.

Repeatedly adding and subtracting 1 from the 'current column' is not
elegant, this should rather be done by consistently measuring only the
indentation from the left border (at offset 0), as a distance, not as an
absolute position.
2021-03-13 00:03:29 +00:00
rillig
8f15c12d1f indent: add 'const', rename variables, reorder formula for tab width
Column counting starts at 1.  This 1 should rather be at the beginning
of the formula since it is thought of being added at the very beginning
of the line, not at the end.

When adding a tab, the newly added tab is added at the end of the
string, therefore that '+ 1' should be at the end of the formula as
well.

No functional change.
2021-03-12 23:27:41 +00:00
rillig
a0306e684f indent: replace 'target' with 'indent' in function names
The word 'target' was not as specific as possible.

No functional change.
2021-03-12 23:16:00 +00:00
rillig
2be5ec967d indent: use consistent indentation for 'else'
Half of the code used -ce, the other half the opposite -nce.

No functional change.
2021-03-12 23:10:18 +00:00
rillig
2603dcca0f indent: make output_string inline
GCC 9.3.0 didn't notice that the argument to this function is always a
string literal, which makes it worthwhile to inline the call.
2021-03-12 19:14:18 +00:00
rillig
51ad939870 indent: add helper functions for doing the actual output
This allows to add debug logging to these few functions instead of all
other places that might output something.

Reducing the possible output formats to a few primitives makes dump_line
simpler, especially the fprintf calls.  It also removes the non-constant
printf string.

The call to output_int may be meant for debugging, as the character 0x80
is unlikely to appear in any real-world code.

No functional change.
2021-03-12 19:11:29 +00:00
rillig
d07ba49995 indent: fix misleading indentation in indent's own code
No functional change.
2021-03-12 18:11:50 +00:00
rillig
61a2b8d236 indent: move code for tokenizing numbers further up
Having it directly below the table makes it easier understandable.

I also tried to omit this function entirely by moving the code into the
initializer itself, but that made the code redundant and furthermore
increased the size of the resulting binary, probably because of the new
relocation records.

No functional change.
2021-03-12 17:46:48 +00:00
rillig
81c22dc68a indent: manually fix indentation
No functional change.
2021-03-12 00:15:34 +00:00
rillig
200ea2d398 indent: reduce indentation of check_size functions
No functional change.
2021-03-11 22:32:06 +00:00
christos
ff8b561e4a remove extra increment; fixes: echo o | sed -e 's/o/\a/' 2021-03-11 22:31:19 +00:00
rillig
c7f0688822 indent: remove redundant cast after allocation functions
No functional change.
2021-03-11 22:28:30 +00:00
rillig
72f722fd46 indent: use consistent array indexing
No functional change.
2021-03-11 22:15:44 +00:00
rillig
a855cd0537 indent: merge duplicate code for reading from the input buffer
No functional change.
2021-03-11 21:47:36 +00:00
wiz
4481672ffb Remove trailing whitespace. 2021-03-11 17:13:29 +00:00
christos
95cea0cc25 Use the same options like m4 (-g turns on GNU, -G turns off GNU) Suggested
by uwe@
2021-03-11 15:45:55 +00:00
christos
d502916eaa Add -G to support GNU extensions 2021-03-11 15:15:05 +00:00
simonb
4e642ca22c Remove Network ATM soft intr queue reporting, we don't have that in the
kernel anymore.
2021-03-10 00:32:15 +00:00
rillig
5edf0687f7 lint: disable check for enum type mismatch in switch statement
This check has been too quick and broke the lint build.  Among others,
lib/libpuffs has -w included in LINTFLAGS, which means that the build
can fail even for new warnings, not only for errors.

libpuffs compares a uint16_t with constants from an unnamed enum type.
Since the enum type is completely unnamed (neither a tag nor a typedef),
there is no way to define a struct member having this type. This was a
scenario that I just didn't consider when I added the check to lint.

For now, disable the new check completely.  The previously existing lint
checks stay enabled, including the one that warns about mismatched
anonymous enum types in the '==' operator, which is very similar to the
now disabled check.
2021-03-10 00:02:00 +00:00
rillig
ee51961c3d indent: extract search_brace from main
No functional change.
2021-03-09 19:46:28 +00:00
rillig
4ca89b5791 indent: extract capsicum code out of the main function
No functional change.
2021-03-09 19:32:41 +00:00
rillig
d60ce5707b indent: rename a few more token types
The previous names were either too short or ambiguous.

No functional change.
2021-03-09 19:23:08 +00:00
rillig
ed0fd2fe70 indent: make token names more precise
The previous 'casestmt' was wrong since a case label is not a statement
at all.

The previous 'swstmt' was overly short, and wrong as well, since it
represents only the 'switch (expr)' part, which is not a complete switch
statement.  Same for 'ifstmt', 'whilestmt', 'forstmt'.

The previous word 'head' was not precise enough since it didn't specify
exactly where the head ends and the body starts.  Especially for
handling the dangling else, this distinction is important.

No functional change.
2021-03-09 19:14:39 +00:00
rillig
d64e8288e4 indent: rename a few tokens to be more obvious
For casual readers it is not obvious whether the 'sp' meant 'special' or
'space' or something entirely different.
2021-03-09 18:28:10 +00:00
rillig
bd056b6eea indent: extract reduce_stmt from reduce
This refactoring reduces the indentation of the code, as well as
removing any ambiguity as to which 'switch' statement a 'break' belongs,
as there are no more nested 'switch' statements.

No functional change.
2021-03-09 18:21:01 +00:00
rillig
6cabfe5cb2 indent: manually indent comments
It's strange that indent's own code is not formatted by indent itself,
which would be a good demonstration of its capabilities.

In its current state, I don't trust indent to get even the tokenization
correct, therefore the only safe way is to format the code manually.
2021-03-09 16:48:28 +00:00
rillig
cf51428060 indent: remove redundant initializer in dump_line
No functional change.
2021-03-08 22:28:31 +00:00
rillig
439ca385c5 indent: move comment about dump_line to column 1
It looked misplaced on the right side since that area is usually
reserved for small remarks, not long explanations.

No functional change.
2021-03-08 22:26:17 +00:00
rillig
a3dc5dfa03 indent: always use braces in do-while loops
Having a 'while' at the beginning of a line looks as if it would start a
loop.  It's confusing when it _ends_ a loop instead.
2021-03-08 22:23:58 +00:00
rillig
8d6721908d indent: split bsearch comparison function
It may have been a clever trick to use the same memory layout for struct
templ and a string pointer, but it's not worth the extra comment and
difficulty in understanding the code.

No functional change.
2021-03-08 21:13:33 +00:00
rillig
68856268b8 indent: inline macro for backslash
No functional change.
2021-03-08 20:20:11 +00:00
rillig
37bc0e6731 indent: convert big macros to functions
Each of these buffers is only modified in a single file.  This makes it
unnecessary to declare the macros in the global header.
2021-03-08 20:15:42 +00:00
rillig
849f856bf6 indent: make it easy to compile indent in debug mode 2021-03-08 19:21:41 +00:00
rillig
4526c76528 indent: fix printing of uninitialized 'token' in debug output 2021-03-08 19:06:48 +00:00
rillig
73071b8322 indent: fix handling of '//' end-of-line comments 2021-03-07 22:11:01 +00:00
rillig
b65a14572e indent: sprinkle a few const
No functional change.
2021-03-07 20:52:11 +00:00
rillig
ec2de7a28a indent: remove redundant parentheses around return value
No functional change.
2021-03-07 20:47:13 +00:00
rillig
17efe8f8c3 lint: move keyword 'continue' over to the other control flow keywords
No functional change since neither rw_jump nor rw_inline_or_restrict is
mentioned in any switch statement, and lint didn't find any other
suspicious enum operations.
2021-03-07 20:40:18 +00:00
rillig
a3d348a576 indent: use named constants for the different types of keywords
This reduces the magic numbers in the code.  Most of these had their
designated constant name written in a nearby comment anyway.

The one instance where arithmetic was performed on this new enum type
(in indent.c) was a bit tricky to understand.

The combination rw_continue_or_inline_or_restrict looks strange, the
'continue' should intuitively belong to the other control flow keywords
in rw_break_or_goto_or_return.

No functional change.
2021-03-07 20:30:48 +00:00
rillig
36dd5d7651 lint: do not allow assignment-expression for constant-expression
It's a funny idea to do something like 'case a = 13:', but since any
compiler will reject this code, there is no point in lint supporting it.

No functional change since everywhere the grammar allows a constant
expression, there is no ambiguity where an assignment could be
interpreted differently.
2021-03-07 20:06:48 +00:00
rillig
53af70edc9 lint: rename grammar rule 'constant'
C99 6.4.4 already defines a grammar rule named 'constant' for an number
literal or an enum constant, so don't use that name for something else.

No functional change.
2021-03-07 19:57:46 +00:00
rillig
4c83646ad2 lint: align names of C grammar rules with C99
No functional change.
2021-03-07 19:46:18 +00:00
rillig
7bd1775519 lint: in strict C mode, warn about initialization with '[a ... b]'
https://gcc.gnu.org/onlinedocs/gcc/Case-Ranges.html
2021-03-07 19:42:54 +00:00
rillig
8c49e96e03 lint: fix off-by-one error in 'case 3...5'
According to the GCC documentation[1], the high end of the range is
inclusive as well, which makes sense since otherwise there would be no
way of specifying a range that includes the maximum representable
number.

Since the range is not used at all in the code, none of the tests could
possibly fail.

[1] https://gcc.gnu.org/onlinedocs/gcc/Case-Ranges.html

No functional change.
2021-03-07 18:02:45 +00:00
rillig
26903e6b19 lint: inline 'case 3...5' in grammar
There was no point in having a separate grammar rule for the '3...' part
of a range expression, it just made the code more complicated than
necessary.

No functional change.
2021-03-07 17:57:52 +00:00
rillig
f4d1bba78c lint: in the name of an array type, list the dimension first
before: array of unsigned int[4]
now:    array[4] of unsigned int

Listing the array dimension first keeps it in contact with the keyword
'array'.  This reduces confusion, especially for nested arrays.
2021-03-07 17:12:41 +00:00
rillig
317f4968ee indent: in debug mode, output detailed token information
The main ingredient for understanding how indent works is the tokenizer
and the 4 buffers in which the text is collected.

Inspecting this debug log for the test comment-line-end makes it obvious
why indent messes up code that contains '//' comments.  The cause is
that indent interprets '//' as an operator, just like '&&' or '||'.  The
sequence '/////' is interpreted as a single operator as well, by the
way.

Since '//' is interpreted as an ordinary operator, any words following
it are plain identifiers, usually several of them in a row, which is a
syntax error.  Depending on the context, the operator '//' is either a
unary operator (no space around) or a binary operator (space around).
This explains why the word 'line-end' is expanded to 'line - end'.

No functional change outside of debug mode.
2021-03-07 11:32:05 +00:00
rillig
bf0defb337 indent: for the token types, use enum instead of #define
This makes it easier to step through the code in a debugger.

No functional change.
2021-03-07 10:56:18 +00:00
rillig
3658a7adc6 indent: use all headers in all files
This is a prerequisite for converting the token types to an enum instead
of a preprocessor define, since the return type of lexi will become
token_type.  Having the enum will make debugging easier.

There was a single naming collision, which forced the variable in
scan_profile to be renamed.  All other token names are used nowhere
else.

No change to the resulting binary.
2021-03-07 10:42:48 +00:00
rillig
280e2f10d4 indent.1: sort options alphabetically 2021-03-06 21:08:08 +00:00
rillig
ff2507050a indent: fix space-tab alignment in indent's own code
These parts are not fixed automatically by indent since they are in box
comments.

No functional change.
2021-03-06 20:30:06 +00:00
rillig
b5348831df lint: warn about enum/enum or enum/int type mismatch in switch
This is something that neither GCC 10 nor Clang 8 do, even though it
seems useful.  Lint didn't do it up to now, but that was probably an
oversight since it is easy to miss the implicit '==' operator in the
switch statement.
2021-03-05 17:10:05 +00:00
simonb
6e33305585 For vmstat -h/-H, calculate the hash element size correctly instead of
assuming that everything that isn't a list is a tailq.  Fixes random
reads from kmem that either fail or return incorrect data for the vcache
hash table.
2021-03-03 08:25:16 +00:00
christos
5d8f515b0c Only specify the dependencies if we are making the copy target. 2021-03-02 14:26:00 +00:00
uwe
d5a7953cd7 Use proper -width instead of the cargo-cult value. 2021-03-02 10:17:25 +00:00
uwe
e6330ecec4 Sort config file flags and command line options. 2021-03-02 10:14:59 +00:00
uwe
b3a91da400 Fix markup so that literals are typeset as literals. 2021-03-02 10:12:33 +00:00
simonb
ec07c80492 Don't show any of the completely and utterly undocumented VTW info if
the feature isn't enabled (by checking net.inet.tcp.vtw.enable).
2021-03-02 01:02:12 +00:00
wiz
d7baa73527 Sort options in synopsis 2021-03-01 21:43:00 +00:00
otis
3128aa54a9 Implement 'E' flag that prevents newsyslog from rotating empty log files. 2021-03-01 21:37:10 +00:00
rillig
d8822d35c5 lint: only warn once about integer constant overflow on 32-bit
Previously, the test msg_056.c warned twice about the integer literal,
but only on 32-bit platforms.  On 64-bit platforms, there was only a
single warning since the integer constant was converted to type
__uint128_t, and this prevented the second warning.  On 32-bit targets,
there is no __uint128_t though.

Fixes part of PR bin/55976.
2021-03-01 00:51:01 +00:00
rillig
523633c84d lint: fix null pointer dereference on parse error
Fixes PR bin/22119.
2021-02-28 22:12:16 +00:00
rillig
d4dabb45ab lint: document where in C99 the behavior of cconv is specified 2021-02-28 20:04:52 +00:00
rillig
6314f2eb79 lint: replace wrong comment with assertion
The broad type of a value is indeed stored in the value itself, in the
member v_tspec.  For nodes that refer to this value, it is redundantly
stored, it always equals tn->tn_type->t_tspec.

After initialization, neither tn->tn_type nor val->v_tspec are modified.
This is not ensured by the compiler but has to be analyzed manually.

No functional change.
2021-02-28 19:24:15 +00:00
rillig
5cc5fe1a43 lint: rename is_nonzero to constant_is_nonzero
The new function name emphasizes that the given node must have the
operator CON.

No functional change.
2021-02-28 19:16:05 +00:00
rillig
6171ad7d6b lint: rename members and access macros of the basic types
Having the measurement unit in the variable name prevents accidental
confusion between bits and bytes, especially since usually the word
'size' means the size in bytes, while 'width' means the size in bits, at
least for integer types.

No functional change.
2021-02-28 18:51:51 +00:00
rillig
2fe9f15dbb lint2: add redundancy checker for comments in source code
Based on the redundancy checker for lint1.

No functional change.
2021-02-28 18:17:08 +00:00
rillig
8f43f8f65c lint2: align comments in the code with actual messages 2021-02-28 17:16:50 +00:00
rillig
a75e3514c2 tests/lint: shorten code for checking redundancy in files
No functional change.
2021-02-28 12:45:47 +00:00
rillig
f963c8e49b tests/lint: keep messages in test files in sync with actual messages
Each of the tests named msg_*.c repeats the template of the message, to
make the test somewhat self-contained when viewed in isolation.

This creates a redundancy, and keeping track of this manually is next to
impossible.  I tried it and failed in 9 cases, even though it has just
been 2 months since I myself created the initial files and I knew all
the time that this redundancy exists.

Be fool-proof for the future by checking this automatically.
2021-02-28 12:40:00 +00:00
mrg
f4968f58f4 disable the rule to copy resize.c since it fails on r/o src trees:
cp: /home/source/ab/HEAD/src/usr.bin/resize/resize.c: Permission denied
2021-02-28 09:04:28 +00:00
rillig
0d0d46fd3a lint: do not warn about constant expressions involving sizeof
These expressions are indeed constant for a specific platform, but on
another platform their value may change.  This makes them unsuspicious
and legitimate for portable code.

Seen in rump_syscalls.c, as 'sizeof(int) > sizeof(register_t)'.
2021-02-28 03:59:28 +00:00
rillig
597058d81f lint: rename parameter in function 'expr'
For symmetry with the function is_constcond_false.

No functional change.
2021-02-28 03:33:18 +00:00
rillig
bdddf63789 lint: document allowed types for bit-fields
No functional change.
2021-02-28 03:14:44 +00:00
rillig
4eef7b9455 lint: extract declare_bit_field from declarator_1_struct_union
No functional change.
2021-02-28 03:05:12 +00:00
rillig
089670865b lint: add type information to 'illegal bit-field type' 2021-02-28 02:45:37 +00:00
rillig
cc09e52e7b lint: extract type_name_of_array from type_name
No functional change.
2021-02-28 02:37:04 +00:00
rillig
ca54ab8987 lint: output precise type information for struct/union/enum
Previously, 'typedef enum { E } name' was output as 'name', which
omitted the information that this was an enum type.  Now it is output as
'enum typedef name'.

Previously, 'typedef struct { int member; } name' was output as 'struct
<unnamed>', which omitted the typedef name.  Now it is output as 'struct
typedef name'.
2021-02-28 02:29:28 +00:00
rillig
2317ba2476 lint: add type information for 'incompatible struct pointers' 2021-02-28 02:00:05 +00:00
rillig
fb1e12220d lint: add type information to enum type mismatch 2021-02-28 01:30:22 +00:00
rillig
4b52b11627 lint: make messages for incompatible pointers more specific
Message 153 didn't state obviously which of the pointer types was the
one before conversion (or cast) and which was the resulting type.

Message 229 didn't have any type information at all.
2021-02-28 01:06:57 +00:00
christos
8e0f075136 add parens 2021-02-28 00:44:58 +00:00
christos
678ed6abf6 no need to specify SRCS 2021-02-28 00:43:24 +00:00
christos
692ead559b remove unused 2021-02-28 00:42:22 +00:00
rillig
b19a5288ba lint: add type information to 'possible pointer alignment problem [135]'
This warning occurs more than 7400 times in a regular NetBSD build, and
without giving any type information, leaves the reader clueless about
what the underlying issue might be.  Add type information since that is
a no-brainer to implement.
2021-02-28 00:40:22 +00:00
rillig
f6dafdc19b lint: skip alignment computation if possible
Testing a global variable is simpler than calling a 20-line function.

No functional change.
2021-02-28 00:28:47 +00:00
rillig
2659adfbbb lint: rename getbound to alignment_in_bits
No functional change.
2021-02-28 00:23:55 +00:00
christos
7c9ac6ddef Don't define the copy rules if we don't have X sources 2021-02-27 22:37:54 +00:00
christos
ec2d1eb5c3 Bump warns 2021-02-27 16:08:08 +00:00
christos
66a2df677d Add a maintainable version of resize, with a copy target and a header that
provides all the glue needed without other X headers.
2021-02-27 15:36:39 +00:00
rillig
99ea8b90fb lint: rename confusing local variable
The variable name rtp is reserved for the type of the right-hand
operand.

No functional change.
2021-02-27 15:26:30 +00:00
christos
e0ecd16b50 PR/56013: Kouichi Hashikawa: Move setutent/setutxent right before the loops. 2021-02-26 02:45:43 +00:00
rillig
d251e04261 make: document the history of bugs in "cmdline overrides global"
For performance reasons, the implementation of the simple rule "cmdline
overrides global" grew into code that is much more complicated than a
straight-forward implementation.  This added complexity made it easy for
bugs to sneak in.
2021-02-23 21:59:31 +00:00
rillig
c82f512123 make: reduce indentation in Var_Parse
The extra condition had been necessary before FStr made memory
management simpler.

The Coverity annotation got out-of-date when the parameter was converted
to FStr since that type is not allocated on the heap, only its inner
members are.

No functional change.
2021-02-23 16:29:52 +00:00
rillig
1029e78d18 make: restructure code in ParseVarname to target human readers
Breaking the loop once for depth == 0 and once for depth == 1 was
unnecessarily confusing, as was the nested 'if'.  Start counting with 0
since there is no reason to start at 1.

Evaluating the common subexpression '*p == endc' is left as an exercise
to the compiler.

No functional change.
2021-02-23 16:14:11 +00:00
rillig
7121bc3d02 make: improve error message for bad modifier in variable expression
The improvement is especially noteable for variable expressions based on
the empty variable, see moderrs.exp:103.
2021-02-23 16:07:14 +00:00
rillig
3429dfe6ac make: add test for confusing error message for bad modifier
In the expression ${:U}, the variable name is empty.  Since these
expressions are generated by .for loops, the error messages for them
must not end with a trailing space.  Putting the variable name in quotes
helps against that.
2021-02-23 16:04:16 +00:00
rillig
985b27d91a make: improve error message for unclosed modifier
Replace "variable specification" with the more modern "variable
expression", reduce the number of parentheses, output more than a single
character for modifiers, make it obvious that in expressions such as
${:Serror}, the "" means a variable name.
2021-02-23 15:56:29 +00:00
rillig
2ede7aa75a make: improve error message for unknown modifier
Back in 1995, the modifiers were all single-character, and it made sense
to print only the first character.  Nowadays, with ':S', ':@var@...@',
'::=' and several others, a little more context is useful to see where
the exact error is.  The actual modifier is still guessed, and the guess
may be wrong as soon as backslashes get involved, but it is still better
than before.
2021-02-23 15:19:41 +00:00
rillig
2b3b250de9 make: remove redundant parameter of ApplySingleModifier 2021-02-23 15:07:58 +00:00
rillig
b5e0384378 make: explain non-obvious code around indirect variable modifiers
No functional change.
2021-02-23 15:03:56 +00:00
rillig
a31532eb14 make: comment on possible inconsistency in handling modifier ':sh' 2021-02-23 14:27:27 +00:00
rillig
face6b4c9d make: note inconsistent parsing behavior in modifier ':_=' 2021-02-23 14:21:45 +00:00
rillig
0b7a08866d make: demonstrate how to undefine variables during evaluation
For a very long time now, I had thought that it would be impossible to
undefine global variables during the evaluation of variable expressions.
This is something that the memory management in Var_Parse relies upon,
see the comment 'the value of the variable must not change'.

After several unsuccessful attempts at referring to an already freed
previous value of a variable, today I discovered how to unset a global
variable while evaluating an expression, which has the same effect.  To
demonstrate that this use-after-free can reliably crash make, it would
need a memory allocator with a debug mode that never re-allocates the
same memory block after it has been used once.  This is something that
jemalloc cannot do at the moment.  Valgrind would be another idea, but
that has not been ported to NetBSD.

Undefining a global variable while evaluating an expression is made
possible by an implementation detail of the modifier ':@'.  That
modifier undefines the loop variable, without restoring its previous
value, see ApplyModifier_Loop.

By the very old conventions of ODE Make, these loop variables are named
'.V.' and thus do not conflict with variables from other naming
conventions.  In NetBSD and pkgsrc, these loop variables are typically
called 'var', sometimes '_var' with a leading underscore, which also
doesn't conflict with the typical form 'VAR' of variables in the global
namespace.  Therefore, in practice these loop variables don't interfere
with other variables.

One case that can practically arise is when an outer variable has a
modifier ':@word@${VAR.${word}}@' and one of the referenced variables
uses the same variable name in the modifier, see varmod-loop.mk 1.10
line 91 for a detailed explanation.

By using the ${:@VAR@@} modifier in a place that is evaluated with
cmdline scope, it is not only possible to undefine global variables, it
is possible to undefine cmdline variables as well.  When evaluated in a
specific make target, the expression ${:@\@@@} can even be used to
undefine the variable '.TARGET', which will probably crash make with an
assertion failure.
2021-02-23 14:17:21 +00:00
wiz
dc9173cfee Fix Fl argument.
From Kouichi Hashikawa in PR 56009.
2021-02-23 07:15:41 +00:00
rillig
ac43378364 make: quote ':S' in error message about missing delimiter 2021-02-23 00:27:47 +00:00
rillig
6b851c0577 make: extract ParseModifier_Match into separate function
No functional change.
2021-02-23 00:25:06 +00:00
rillig
54bec0ad03 make: add context information to error message about ':range' modifier 2021-02-23 00:15:22 +00:00
rillig
39ed2c5332 make: fix local variable name for parsing arguments
The variable name 'arg' was misleading since after a successful
TryParseTime, it would no longer point to the argument of the variable
modifier, but to the _end_ of the argument.  To reduce confusion, use p
instead, like everywhere else.  This name is less specific, which is
still better than a wrong name.
2021-02-23 00:11:07 +00:00
rillig
0c470650a5 make: add quotes around variable name in an error message 2021-02-23 00:04:48 +00:00
rillig
ba8edc0160 make: add test for confusing double space in error message 2021-02-22 23:59:43 +00:00
rillig
3d4c16a2c1 make: reorder code in ModifyWords
No functional change.
2021-02-22 23:46:03 +00:00
rillig
cc8e99da10 make: use more common parameter order for VarSelectWords
No functional change.
2021-02-22 23:42:29 +00:00
rillig
236224f584 make: make ModifyWord_Subst a little easier to understand
Addition is easier than subtraction, and the expression 'word + wordLen'
obviously means 'the end of the word', which was not as easy to spot
before.

No functional change.
2021-02-22 23:39:24 +00:00
rillig
49c1680d06 make: remove freestanding freeIt variables
These variables all belong to a string variable.  Connect them using
FStr, which reduces the number of variables to keep track of.

No functional change.
2021-02-22 23:21:33 +00:00
rillig
4acdc7c93b make: remove boolean parameter and return type from VarFreeEnv
There was only a single case where this parameter was false.  Inline
that case.  That was the only case that needed the return value, so
remove that as well.
2021-02-22 22:55:43 +00:00
rillig
626c9c496d make: add a few open questions to Var_SetWithFlags 2021-02-22 22:34:04 +00:00
rillig
db1760d592 make: do not expand variable name from the command line twice in a row
When

1.  there is a global variable containing a dollar in its expanded name
    (very unlikely since there are lots of undocumented edge cases that
    make variable names containing dollar signs fragile), and

2.  after that (unlikely since that requires .MAKEFLAGS instead of a
    normal command line)

3.  there is a command line variable of the same name (again very
    unlikely since that variable name would contain a dollar sign as
    well in the expanded form),

the global variable would not be undefined as promised by the comments
since its name was expanded once more than intended.

Because of the two 'very unlikely' above, this edge case hopefully does
not affect any practical use cases.

Note that this is not about VAR.${param} (which has a dollar sign in its
unexpanded form), but about the case where param itself would expand to
a dollar sign, such as after param=$$.
2021-02-22 22:26:50 +00:00
rillig
fbed66c0a8 make: add test for cmdline variables overriding global variables 2021-02-22 22:04:28 +00:00
rillig
a1ab648646 make: extract ExistsInCmdline from Var_SetWithFlags
No functional change.
2021-02-22 21:43:57 +00:00
rillig
ae25c3cc92 make: save a hash map lookup when defining a cmdline variable
This is a preparation to extract the code for exporting a cmdline
variable.  That code differs in several details from the other code in
ExportVar.

No functional change.
2021-02-22 21:30:33 +00:00