against the directory name of the makefile being processed.
Shortens a lot of pathnames (especially in pkgsrc) and significantly
reduces the number of duplicate entries in .MAKE.MAKEFILES
are first processed, not for every line of every loop inside the file.
When (attempting) to check for duplicate names, use the spaces between names
as separators rather than using strstr() - which is slow and wont find
duplicates after an incorect early match.
However the duplicate check is rather broken (especially for pkgsrc) where
names like ../../devel/atk/../../devel/glib2/../../devel/gettext-lib/ \
../../devel/gettext-lib/../../mk/buildlink3/../../mk/bsd.prefs.mk
which need (some form of) realpath() processing.
'cd pkgsrc/xaa/gtk2; make show-var VARNAME=.MAKE.MAKEFILES' shows the problem.
The issue seems to have been present for some time, only showed up
when running unit-tests on SunOS.
Make sure we get an error message, but otherwise behave as before.
dependency line.
This makes the line "foo:;echo bar" not try to execute "cho bar", and the
line "baz:;" not treat the rest of the makefile as commands.
Pointed out by Takahiro Kambe on current-users.
and also include files don't have mismatched .if/.endif
It has been suggested that make used to have this test, but I can't
remember removing any code that might have performed it.
Instead of copying the file byte by byte into an expanding line buffer, just
scan the file buffer for (unescaped) newlines.
Escaped newlines (etc) are processed by modifying the file buffer later.
Speeds up 'nbmake obj' by a factor of 2.
I suspect it will do wonders for parts of pkgsrc as well, since it largely
eliminates cpu cycles from the false part of .ifs.
Commons up the code for reading files and buffers, and allows a further
change where we don't copy the data out of the buffer when generating 'lines.
This will speed up makefile parsing (esp .if clauses).
ParseTraditionalInclude() into a separate routine.
Fix the 'use after free' and 'free on item not malloced' that got added
to ParseTraditionalInclude() in Feb 2006 (rev 1.111).
Kill the 'PTR' struct and put both its members into IFile.
Remove the parameter from ParseEOF(), 1 of the 3 calls passed the wrong value!
Fortunately another test stopped anything nasty happening, we'll use that
test instead.
into every 'gnode' (aka target), instead just copy a pointer to the string
and avoid freeing the original name when we close the file.
I can't imagine any makefile set where this gives a larger footprint!
the short parts of some conditionals first.
First step towards speeding up the parsing of makefiles (esp. the unwanted
parts of .if clauses).
There should be no changes to the logic.
- Only test 'printVars' once.
Has side effect of not trying to find the 'main' target for '-V varname'.
- Only reap the single child process in Cmd_Exec(),
I think this is for 'xx != cmd' so probobably no other children should exist.
- Don't read and parse .depend if '-r' and '-V varname' are set.
I suspect that .depend shouldn't affect the output of any -V command...
It doesn't need a two-dimensional array to remember the states of .if lines.
It would be even simpler if we didn't try to detect .else and .elif lines
that follow .else lines.
Unfortunately this isn't the code that is stupendously slow...
Document the -d-flags and -dFfilename.%d options.
Mention MAKEFLAGS at the top, reword to make it clear that makefile and
Makefile are only used if no -f makefile is given.
This gives a considerable speedup in the processing of .WAIT and .ORDER.
Both .WAIT and .ORDER stop both the commands of the node, and its dependant
nodes being built until the LH nodes are complete.
.WAIT only applies to the dependency line on which it appears, whereas
.ORDER applies globally between the two nodes.
In both cases dependant nodes can be built because other targets need them.
make now processes the target list left to right, scheduling child nodes
as they are needed to make other nodes (instead of attempting to generate
a bottom-up dependency graph at the start). This means that 'make -j1'
will tend to build in the same order as a non-parallel make.
Note that:
all: x y
x: a .WAIT b
y: b .WAIT a
does not generate a dependency loop.
But
x: y
.ORDER y x
does (unless something elswhere causes 'y' to be built).
replace the %d with getpid().
Allows the trace files for different make process is a big recursive make
to be written to different files - and have a slight chance of being useful.
globbing.
The old behaviour was the perform variable expansion and globbing on the
output of both the variable expansion and globbing. Which allows some very
strange behaviour if, for example, globbed filenames contain $ symbols.
Unconditionally add new nodes from these expansions even if the names are
already children. The .WAIT code needs the order of children preserved.
Rename 'allsrc' to 'curSrcs' so that the variable passed to ParseDoSrc (etc)
has the same name as the local in the calling routine, rather than a name
that seems to imply a more global and persistent list.
loop continue part. Remove the 'free(line)' from teh end of the loop body,
change the gotos into continues and collapse out some 'else if's.
All in the name of code transparency. No functional changes.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.
entries, so allocate at startup.
Use an 'int jobPipe[2]' within the job structure, and create pipes directly
into it. Common up the code that creates all the pipes - making them all
non-block on the read side in the process.
Call Job_CatchChildren() directly from Job_CatchOutput() so that it only
gets called when a child actually exits.
NB: Something causes a 'pregnant pause' if (for example) you call 'nbmake obj'
in src/tools. Introduced between netbsd 3 and 4.
shell. To use this, build with -D_PATH_DEFSHELLDIR=\"/path/to/dir\"
-D_BASENAME_DEFSHELL=\"shell\".
* Change the order of entries in shells[]. Now DEFSHELL defaults to 0,
and shells[0] describes the default shell. This will be "sh" in the
usual case.
* If _BASENAME_DEFSHELL is defined, insert an additional entry above
"sh" in the shells[] array, making this new entry shells[0]. The
new entry is assumed to refer to an sh-compatible shell with a
non-standard name. (Tested using _PATH_DEFSHELLDIR="/usr/pkg/bin" and
_BASENAME_DEFSHELL="bash".)
* In the shells[] entry for "sh", test defined(MAKE_NATIVE) &&
defined(__NetBSD__) to decide whether we can use the "q" flag.
OK sjg, christos
messages whan make itself is suspended (ie by ^Z) before make actually
suspends, supress the messages during this sequence.
This means we don't care that they would be output after the suspend
and we can stop attempting to reap child status from withing the signal
handler (which doesn't work for recursive parallel makes).
The code simplification means that we can remove much of the code that
blocked signals - since the signal handlers (expect that for ^C and friends)
now do almost no work.
If there are any undead ones set a flag so we don't report the 'Child (pid)
not in table' message when they die - it is impossible to (portably) find
the childrens pids.
This happens when make is run as 'make -f- ... <<EOF' and the shell uses
a child of (what will be) make to write the data into a pipe.
- Send each type of signal to its own handler.
- Only call JobFinish when a process exits, in particular don't 'fake up'
'exitstatus' for jobs being continued, nor call it for suspends.
- When a job is stopped, use an entire variable to remember the fact, so
we know we need to send a SIGCONT. Don't change any other state.
- In order to report '*** [job3] Suspended' before we suspend ourselves we
have to call waitpid() from the signal handler - where we don't want to
process job termination events. Save the exit status and process later.
The code now handles:
- jobs that suspend themselves
- jobs exiting while suspended
- jobs that don't actually suspend at all
Hoewever it still does printfs() from the signal handler, and I haven't yet
stopped it thrashing the signal mask.
with a table that is malloced with 'maxJobs' entries.
Add a 'job_state' field to the Job type that exactly follows which of
the old lists the job was on (or not).
Change all the code that scanned the lists to scan the array.
No logic changes in this commit.
(Soon we'll no longer need to lock out signals for the changes to job
statuses that are done from signal handlers now that there is no linked list.)
This cuts Var_Parse in half! and allows the modifier logic to
be used recursively - when getting modifiers via variables.
Add new unit-test, to check that certain error cases are handled
correctly.
compiler error when building buf.c with -O2, so reduce optimization
level to -O1 for this single file.
To be documented in docs/HACKS, discussed with skrll.
the expectations of Lst_Find*. This way we only read the first sys.mk
found via sysIncPath.
At the same time we need to add a ReadAllMakefiles() for the case
where multiple -f makefile args are provided (uncommon, but documented).
functions work. When they allocate storage that needs to be freed, instead
of setting a boolean, set the pointer to be freed. Plug some more memory
leaks found by inspection.
include statements so that if a variable expands to more than one file
name make will "do the right thing".
- Add additional debug print
Reviewed by christos.
Looks like it was intended as a minor (and pointless) optimisation to
remove a free() malloc() pair.
Make he comment about the stoppedJobs list more correct.
It isn't clear that it ever worked, if it did it has almost certainly
bitrotted in the last 12 years. I'm not even sure all the required
components were present.
I suspect it was written to attempt to use a 'farm' of diskless sun3s.
In any case the apparant random assignment fo jobs to other systems doesn't
actually seem like a good idea!
Things like 'distcc' han be used to help slow systems run native builds.
Removing this code also simplifies make, and should let me speed up some of
its processing - without worrying about bitrotting it further.
push a byte through the (now badly named) exit_pipe and call JobRestartJobs()
from the main code path when poll() wakes up.
Part of a plan to remove JobSigLock() and the zillions of system calls
it does.
hand side of the .WAIT, except when the recursive interpretation would
cause a cycle in the dependency graph.
Discussed in tech-toolchain. Reviewed by christos, sjg.
De-optimise the getting of a job token so we don't re-order the job
list when there are no tokens.
This might have helped etc/Makefile, but isn't enough.
of trying to de-jobify the make.
You can now put .NOTPARALLEL in a submake of a recursive make (where it is
using a job-token pipe from the outer make and have it only run a single job.
You can also specify .NOTPARRALEL in the root makefile of a large recursive
make and have the submakes run multiple commands.
Add some diagnostics printfs (enabled with -dp) to the parser.
the first one for each make. This significantly speeds up the detection
of errors in other branches of the make (ie those running in a different
make process). The cost of reading and writing a byte from the pipe
should be insignificant.
Defer replacing job tokens until we've decided there is an error.
If we detect an error in another branch of the make, then call Fatal(),
setting 'aborting' and failing to return a token leads to infinite loops.
Now parallel makes actually stop with the failing command on the screen.
job_pipe and collect another one for the next job.
If we are aborting, remove all the 'normal' job tokens and add an 'error' one.
If we get an 'error' token, remove any other tokens, re-insert the error
token and exit (with error 'cos that is easier).
Add the current pid to some of the DEBUG(JOB) traces.
Combined effect is that parallel makes actually stop some fairly shortly
after an error, rather than running on long enough to fill the scrollback.
output in parallel makes.
After all with -s you wouldn't know the command for a non-parallel make.
Makes (sic) the output of parallel NetBSD build fathomable.
loop expansions, when the expanded variable ends in backslash and
the backslash is the last character on the line. While this fix is
ugly (detect the condition and append a space), it is the least
intrusive for now.
to document the behaviour that is currently in use (the "./obj" and
"/usr/obj/`pwd`" behaviour).
Hopefully the existing .OBJDIR behaviour is clearer now.
./obj.${MACHINE}
./obj
/usr/obj/${PWD}
The rules for the default .OBJDIR setting are now simplified to
(and documented as) trying the chdir to the following
(if the appropriate variable is defined):
${MAKEOBJDIRPREFIX}${.CURDIR}
${MAKEOBJDIR}
${.CURDIR}
.OBJDIR can be overridden in the makefile.
<bsd.obj.mk> uses this to provide the "culled" .OBJDIR semantics
for NetBSD's /usr/src builds.
MAKEOBJDIRPREFIX & MAKEOBJDIR still can only be provided
in the environment or on make(1)'s command line.
Per discussion on tech-toolchain.
This should reduce a lot of lossage people have experienced over
the years with various .OBJDIR setups.
mode, use the ignErr template for the command as shell doesn't like an empty
construct of the form { } || <something>. Fixes build breakage on cats
distrib where a command ends up expanding to nothing.
* Explicitly goto default_case for unknown chars encountered after
various : modifiers, rather than multiple FALLTHRUs.
* Appease gcc -Wuninitialized for sv_name and v_ctxt.
Discussed with sjg.
ignoring suffix-specific path search. So if a node was marked .MADE,
then suffix rules would not be applied to it, and we would look for
the file only in the default path, not the suffix-specific path.
XXX: Now that we looked for the suffix, we can save it in the GNode,
but we don't do this yet.
Its not posix compliant and serves very little purpose.
With this change compat and jobs modes are consistent wrt how
they treat each line of a script.
Add support for the '+' command line prefix as required by posix.
Lines prefixed with '+' are executed even when -n is given.
[Actually posix says they should also be done for -q and -t]
PR:
Reviewed by: jmc
Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().
It's now possible to build make on some hosts with: cc *.c */*.c
Notes:
* The immediately previous version of make errored out on --,
which is what needs fixing.
* Historic bsd make silently ate -- and continued to process
options, so this is a behavior change from that, too, but presumably
there is more to gain in fixing it than in being bug-compatible.
dsl