Commons up the code for reading files and buffers, and allows a further
change where we don't copy the data out of the buffer when generating 'lines.
This will speed up makefile parsing (esp .if clauses).
- Only test 'printVars' once.
Has side effect of not trying to find the 'main' target for '-V varname'.
- Only reap the single child process in Cmd_Exec(),
I think this is for 'xx != cmd' so probobably no other children should exist.
- Don't read and parse .depend if '-r' and '-V varname' are set.
I suspect that .depend shouldn't affect the output of any -V command...
This gives a considerable speedup in the processing of .WAIT and .ORDER.
Both .WAIT and .ORDER stop both the commands of the node, and its dependant
nodes being built until the LH nodes are complete.
.WAIT only applies to the dependency line on which it appears, whereas
.ORDER applies globally between the two nodes.
In both cases dependant nodes can be built because other targets need them.
make now processes the target list left to right, scheduling child nodes
as they are needed to make other nodes (instead of attempting to generate
a bottom-up dependency graph at the start). This means that 'make -j1'
will tend to build in the same order as a non-parallel make.
Note that:
all: x y
x: a .WAIT b
y: b .WAIT a
does not generate a dependency loop.
But
x: y
.ORDER y x
does (unless something elswhere causes 'y' to be built).
replace the %d with getpid().
Allows the trace files for different make process is a big recursive make
to be written to different files - and have a slight chance of being useful.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.
entries, so allocate at startup.
Use an 'int jobPipe[2]' within the job structure, and create pipes directly
into it. Common up the code that creates all the pipes - making them all
non-block on the read side in the process.
Call Job_CatchChildren() directly from Job_CatchOutput() so that it only
gets called when a child actually exits.
NB: Something causes a 'pregnant pause' if (for example) you call 'nbmake obj'
in src/tools. Introduced between netbsd 3 and 4.
the expectations of Lst_Find*. This way we only read the first sys.mk
found via sysIncPath.
At the same time we need to add a ReadAllMakefiles() for the case
where multiple -f makefile args are provided (uncommon, but documented).
It isn't clear that it ever worked, if it did it has almost certainly
bitrotted in the last 12 years. I'm not even sure all the required
components were present.
I suspect it was written to attempt to use a 'farm' of diskless sun3s.
In any case the apparant random assignment fo jobs to other systems doesn't
actually seem like a good idea!
Things like 'distcc' han be used to help slow systems run native builds.
Removing this code also simplifies make, and should let me speed up some of
its processing - without worrying about bitrotting it further.
hand side of the .WAIT, except when the recursive interpretation would
cause a cycle in the dependency graph.
Discussed in tech-toolchain. Reviewed by christos, sjg.
of trying to de-jobify the make.
You can now put .NOTPARALLEL in a submake of a recursive make (where it is
using a job-token pipe from the outer make and have it only run a single job.
You can also specify .NOTPARRALEL in the root makefile of a large recursive
make and have the submakes run multiple commands.
Add some diagnostics printfs (enabled with -dp) to the parser.
job_pipe and collect another one for the next job.
If we are aborting, remove all the 'normal' job tokens and add an 'error' one.
If we get an 'error' token, remove any other tokens, re-insert the error
token and exit (with error 'cos that is easier).
Add the current pid to some of the DEBUG(JOB) traces.
Combined effect is that parallel makes actually stop some fairly shortly
after an error, rather than running on long enough to fill the scrollback.
./obj.${MACHINE}
./obj
/usr/obj/${PWD}
The rules for the default .OBJDIR setting are now simplified to
(and documented as) trying the chdir to the following
(if the appropriate variable is defined):
${MAKEOBJDIRPREFIX}${.CURDIR}
${MAKEOBJDIR}
${.CURDIR}
.OBJDIR can be overridden in the makefile.
<bsd.obj.mk> uses this to provide the "culled" .OBJDIR semantics
for NetBSD's /usr/src builds.
MAKEOBJDIRPREFIX & MAKEOBJDIR still can only be provided
in the environment or on make(1)'s command line.
Per discussion on tech-toolchain.
This should reduce a lot of lossage people have experienced over
the years with various .OBJDIR setups.
Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().
It's now possible to build make on some hosts with: cc *.c */*.c
Notes:
* The immediately previous version of make errored out on --,
which is what needs fixing.
* Historic bsd make silently ate -- and continued to process
options, so this is a behavior change from that, too, but presumably
there is more to gain in fixing it than in being bug-compatible.
No functional change under NetBSD.
Restarting a getopt(3) loop is an extension to the posix getopt(3)
behavior and is not portable.
Fixes tools build (tools/groff) under Cygwin.
- new dir.c function: Dir_FindHereOrAbove:
Search for a path in the current directory and then all the directories
above it in turn until the path is found or we reach the root ("/").
- add hooks to use it in main.c for -m and syspath (compiled in
_PATH_DEFSYSPATH and $MAKESYSPATH).
- updated man page
objdir) try opening the file from curdir and if that fails try from objdir.
This way .depend files get picked up via their full path names rather than
just .depend so vars like PARSEDIR, PARSEFILE are then set correctly. This
fixes PR#13289 reporting incorrect pathnames for .depends with errors in
them
the .DOTLAST primitive by a boolean variable with the same name, this whole
mechanism was broken ... it doesn't save much stat calls but it was wrong.
Thanks to Jason Thorpe for the other shadow-variable fixing patches he
made.
Instead of skipping the PWD check entirely, add a skip of the PWD override
if MAKEOBJDIR is set and contains a variable transform ($). This has
similar problems to what happens if MAKEOBJDIRPREFIX is set.
breaks too many situations, including MAKEOBJDIR with a :C,foo,bar,
transform in many cases. It's ambiguous and unreliable, as the comment
above that code always indicated. In order to have reliable objdirs, they
need to work the same way Every Time.
(Note that taking this out is not a performance hit; we were already doing
the getcwd() call first. So the getenv("PWD") didn't increase performance.)
* Replace chdir_verify_path() with Main_SetObjdir(), which can be called
externally, and can take a "const char *". (There's a lot of non-const
"char *" passing around in var.c of what should be const strings....)
* Rewrite the initial "find my .OBJDIR" code to make use of the new
function. This still functions as it had in the past, but the comment
above this block was changed to reflect reality: if MAKEOBJDIRPREFIX
or MAKEOBJDIR are set in the environment, then *only that value* is
tried; make does not fall back to obj.MACHINE, obj, and /usr/obj/`pwd`
as it would without these env vars set.
* Add a new special target, .OBJDIR:, which when parsed will cause make to
change to a new object directory and reset .OBJDIR, and PWD in the
environment. This will allow some makefiles (mainly, src/tools)
to override the default objdir semantics in order to add custom logic.
(This splits out the "default system include paths" into its own Lst
variable, and uses it only if sysIncPath is empty. This allows sysIncPath
to be filled in by the Makefile itself.)
We now just list the names of such variables in .MAKEOVERRIDES.
When we come to export MAKEFLAGS we quote the value of each exported variable
using :Q, using: ${.MAKEOVERRIDES:O:u:@v@$v=${$v:Q}@}
The :O:u suppresses duplicate names.
Also modifed Parse_DoVar to re-export MAKEFLAGS whenever .MAKEOVERRIDES
is assigned to so .MAKEOVERRIDES+= PATH will export PATH=${PATH:Q}
to the environment, while .MAKEOVERRIDES= will disable export of VAR_CMD's.
From Var_Set: We actually want the equivalent of
.MAKEOVERRIDES:= ${.MAKEOVERRIDES:Nname=*} name='val'
clearing the previous value for name is important, since
doing simple duplicate suppression does not handle:
$ make FOO=goo
which then runs a sub-make with FOO=boo
the commands from that sub-make should see just FOO=boo.
via MAKEFLAGS. Instead of appending them directly to .MAKEFLAGS, put
them in .MAKEOVERRIDES (and ensure they are quoted). This is now done
in Var_Set when it exports VAR_CMD's.
Use ExportMAKEFLAGS() to export MAKEFLAGS, using the combined content
of .MAKEFLAGS and .MAKEOVERRIDES (with duplicate supression).
If .MAKEFLAGS is assigned to in a Makefile, ExportMAKEFLAGS is called again.
This allows a line like:
.MAKEOVERRIDES=
to effectively stop the exporting of the command line vars in MAKEFLAGS.
1. make -dx turns on DEBUG_SHELL which causes sh -x to be used where
possible.
2. PrintOnError() is now called when make is stopping due to an error.
This routine reports the curdir and the value of any variables listed
in MAKE_PRINT_VAR_ON_ERROR.
3. Variables set via command line, are propagated to child-makes via
MAKEFLAGS. This behaviour appears to be necessary for POSIX (according
to the GNU folk anyway).
4. Do not reset MAKEFILE when reading ".depend" as this rather eliminates the
usefulness of ${MAKEFILE}.
5. Added ${.newline} as a simple means of being able to include \n in the
result of a :@ loop expansion.
6. Set ${MAKE_VERSION} if defined. Need to come up with a useful value.
Reviewed: christos
Add -N flag to *really* not execute any commands (useful when using
the -d flags to debug usr/src/Makefile)
Document -N
Update documentation of -n to mention that it still executes commands
for targets marked .MAKE so that the -N/-n distinction is clear.
use -j; all make's in a recursive build cooperate to limit the total
number of jobs, using a token-passing scheme.
The current token passing algorithm is similar to the one implemented
by gmake; there is a single pipe which is inherited through the entire
process hierarchy; tokens are obtained by reading a byte from the
"read end" of the pipe, and are returned by writing them to the "write
end". This exact algorithm is likely to change in the future.
Implementation details:
- Use the new trace facility to allow measurement of the
effectiveness of different token-passing schemes
- Get a token in MakeStartJobs(), return it in Make_Update()
- Eliminate Job_Full() and the jobFull global since they are
redundant with token system.
- Add an "internal" -J option (to pass the token pipe fd's down to
submakes) and a -T option for tracing.
- Change how compatMake is forced so that -j means something when
inherited by submakes.
- When waiting for a token, poll the token-passing pipe as well as
the output pipes of existing jobs.
is an empty list, use DEFSYSPATH.
The current behaviour may have been useful when DEFSYSMK was an
absolute path (hasn't been the case since 1996), but right now
make -m /no/such/dir will fail to find sys.mk and die.
Firstly, we ignore getenv("PWD") if MAKEOBJDIRPREFIX is set so that we always
get the same value for .CURDIR regardless of how make was invoked.
Second, when executing a command we check if it is ${.MAKE} or ${.MAKE:T}
without a preceeding chdir, if so we insert a chdir(${.CURDIR}) so that
the Makefile will be found by the child make. Note that this behaviour is
dissabled if MAKEOBJDIRPREFIX is not set or if NOCHECKMAKECHDIR is set.
See the comments in main.c for more detail.
With these two changes, one can successfully build usr/src using MAKEOBJDIRPREFIX
allowing the src to be mounted from a CD-ROM.
someone wants to do freaky stuff with $MAKEOBJDIR.
Initialize the Var system and set .CURDIR, MACHINE and MACHINE_ARCH
before chdir_verify_path() is called.
Build a list of `cohorts' as before, but do *not* link each one into all the
parent nodes; instead, copy the `cohort' lists into the stream of targets to
be built inside Make_ExpandUse(). Also do the attribute propagation as a
separate pass after parsing.
This eliminates several O(n^2) algorithms.
This revealed another long standing problem with pmake's port to bsd.
.MAKE was not set as the manual page states. Set it and remove another
typo in my last commit.
dsl