Commit Graph

49 Commits

Author SHA1 Message Date
Alexander von Gluck IV
50c891839a bootstrap: Fix behaviour after default haikuporter mode change 2017-07-02 18:23:30 -05:00
Adrien Destugues
fda0bf77fb Revert "repo rework: Re-add compatibility url field"
Revert "repo rework: Remove stubs; Breaks repo compat."
Revert "repo rework: Remove need for repos to be self-aware"

This reverts commit a2b2f4d642.
This reverts commit 602076ef82.
This reverts commit 5ffaf72c8a.

These changes break the build on Haiku and the ability to create repo
mirrors, for the lack of a replacement for the URL (an UUID was evoked
on the mailing lists, but not implemented).

We are due for a release soon, please don't break the build.
2016-12-10 12:55:56 +01:00
Alexander von Gluck IV
5ffaf72c8a repo rework: Remove need for repos to be self-aware
* See #12917 for details.
* Squashed to one commit to make revert easy if we
  run into any issues.
* pkgman is now pre-attached to the 'current' repo
  version within nightly images so they can be updated
  by default.
* This shouldn't impact us keeping older sets of package
  versions by commit hash for building older hrevs.
* There are XXX stubs with "Kill me". These will need
  to be dropped after users are given sufficent time to
  upgrade. We're dropping a previously required field (url)
  so making this a slowish roll out.
* Makes the repos a lot less restrictive which should
  help PM package building automation be a bit easier.
* Once this stuff smooths out, we'll add UUID's to the
  repo definitions for duplicate repo detection.
2016-12-02 00:41:06 -06:00
Jonathan Schleifer
c9dd05ff40 configure: Add variable to see if we're bootstrapping 2014-10-26 21:07:57 +01:00
Oliver Tappe
5399d1df38 Fix (hopefully) bootstrap build with HAIKU_NO_DOWNLOADS=1
* With HAIKU_NO_DOWNLOADS=1, the check against existing package files
  in the download folder should only be done in the phase that is
  adding packages to be put onto the resulting target image, not in the 
  phase that is adding the bootstrap packages (as here those packages 
  will be *built*, not downloaded).
2014-09-14 14:30:56 +02:00
Oliver Tappe
e29a6c4cd4 Introduce stage0 into the bootstrap process.
* This stage builds the gcc packages to get the shared syslibs, which
  only requires the Haiku glue code.
* Add separate declaration section for stage0 packages to
  HaikuPortsCross repository files.
* For the bootstrap_stage0 platform, fall back to the gcc headers
  provided by the cross-compiler.
2014-08-13 13:34:26 +02:00
Oliver Tappe
2e477cf688 Simplify handling of package names.
* No longer apply somewhat crude mechanism for converting a package name
  into a corresponding port name - haikuporter by now supports package
  names directly.

This fixes a problem when trying to pass 'gcc_syslibs_devel' to
haikuporter, which only ever saw 'gcc_syslibs'.
2014-08-13 13:34:07 +02:00
Oliver Tappe
938bf9fad2 Add more missing 'local's to variable declarations. 2014-08-13 13:32:43 +02:00
Oliver Tappe
2ae3c64943 Improve IsPackageAvailable()
* When adjusting the package name for the secondary architecture,
  it is unclear where exactly in the package name the architecture
  specifier is. To remedy, we try all possible positions until
  we find the package (or there are no other possibilities).
2014-07-31 10:49:36 +02:00
Oliver Tappe
1b4510eebf Put haiku repository onto image.
* Now that system updates seem to work properly, put the haiku
  repository config and cache file onto the image automatically.
* Adjust URL of haiku repository (it is currently redirected
  to some other URL at download.haiku-os.org, but that will be
  changed later).
2014-07-26 00:35:53 +02:00
Oliver Tappe
6f32bafa70 Use a specific packager for continuous integration builds. 2014-07-03 18:12:07 +02:00
Oliver Tappe
4e37addd2f Add HAIKU_PORTER_EXTRA_OPTIONS variable.
* This can be used to add arbitrary options to invocations of 
  haikuporter.
2014-06-23 17:19:45 +02:00
Oliver Tappe
d1b21b4ad3 Improve build feature handling with HAIKU_NO_DOWNLOADS=1
* Adjust the respective rules such that with disabled downloads, only
  packages already available in the downloads folder will be considered
  as available build features.

This way, the build system will not for instance try to build
<kdebug>qrencode after a bootstrap, as that package is not yet
available.
2014-05-01 19:45:19 +02:00
Oliver Tappe
aa2e5eca78 Add new configuration option --no-downloads.
* If --no-downloads has been given, Haiku will be built without
  trying to download anything, all required packages need to be put
  into the download folder manually (the build will stop on missing
  packages).
* As the required HaikuPorts repository can't be downloaded in this
  mode, a local repository is created during the build, which only
  contains the packages available in the downloads folder.

This is useful for building Haiku completely from source.
2014-05-01 13:03:43 +02:00
Oliver Tappe
8c54af563c Whitespace cleanup in root and 'build' folder.
* Removed trailing whitespace and empty lines at top of files.
2014-05-01 12:55:41 +02:00
Jonathan Schleifer
fe7d7c27fc RepositoryRules: Add option to now download any packages
This is useful if Haiku should be built using self-built packages.
2014-04-03 15:55:44 +02:00
Oliver Tappe
a8c098a3ba Replace HAIKU_{BOOTSTRAP,MINIMUM}_BUILD with HAIKU_BUILD_TYPE.
* Instead of separate variables, HAIKU_BUILD_TYPE is set to one of
  'bootstrap', 'minimum' or 'regular'.
* Adjust uses of HAIKU_BOOTSTRAP_BUILD accordingly.
2014-03-22 20:24:07 +01:00
Oliver Tappe
1d50d90fbe Drop extra packages and note which no longer applies.
* now that haikuporter resolves the dependencies correctly, there's no
  need to add texi2html and htmldoc explicitly
2014-03-16 14:48:30 +01:00
Oliver Tappe
72e677911f Make set of source packages for bootstrap-image configurable.
* Add new build profile 'minimum', which defines a minimum set of
  packages.
* Introduce HAIKU_BOOTSTRAP_SOURCES_PROFILE and let it default to
  '@minimum-raw'. This can be overruled in UserBuildConfig, setting
  it to '@release-raw' will cause all source packages required for a
  full release to be put onto the bootstrap-image.
* Add new image HaikuImageMinimum, which is meant to define the minimum
  useful Haiku image (yeah, I know that's vague).
* Add HAIKU_MINIMUM_BUILD, which indicates that HaikuImageMinimum should
  be used (it would be better to merge this with HAIKU_BOOTSTRAP_BUILD
  into something like HAIKU_BUILD_TYPE)
* Cleanup duplicate references to basic packages - those are now added
  by the topmost Jamfile (no longer referenced by the build profiles).
2014-03-16 01:17:55 +01:00
Oliver Tappe
5c9657b7e5 Fix creation of haikuports.conf for bootstrap image.
* comments in here-documents don't work (the shell filters them out),
  so we echo the comment line explicitly
2014-02-23 18:46:18 +01:00
Oliver Tappe
68f8b94e2a Drop --remote-user configuration option again.
* as Ingo has pointed out, the remote user settings doesn't
  relate to the build configuration at all, so setting the
  remote user via HAIKU_REMOTE_USER in UserBuildConfig or
  via shell environment is the way to go
* additionally: drop debug output
2014-01-22 17:31:26 +01:00
Oliver Tappe
86d7e28306 Add support for specifying remote ssh login user.
* add option --remote-user to configure, which sets HAIKU_REMOTE_USER
* add evaluation of HAIKU_REMOTE_USER variable when ssh-ing
  into git.haiku-os.org
2014-01-22 09:36:26 +01:00
Ingo Weinhold
be5e6fefdf Add repository config for the Haiku repository to the image
* HaikuRepository rule: Create the repository config.
* HaikuImage: Add the repository config for the Haiku image. The
repository cache is not added, though (it would only be available, if
the repository had been built before).

Implements #10287. The Haiku repository is now available in Haiku by
default.
2014-01-19 00:49:11 +01:00
Ingo Weinhold
f0b99b54bb RepositoryConfig rule: Make a bit more generic
Instead of the repository the URL can now be passed. It can use the
"$version" placeholder, which will be replaced by the content of the
given version file. If the URL is not given, the one from the repository
info will be used.
2014-01-19 00:49:11 +01:00
Ingo Weinhold
12c19e6362 Move creating the Haiku repo info to jam
* PreprocessPackageInfo rule: Pull out new rule
PreprocessPackageOrRepositoryInfo which does the sed substituation and
optionally the filtering through the C preprocessor.
* HaikuRepository rule: Generate the repository info file (from the
given template). No longer do that in the build_haiku_repository
script.
2014-01-19 00:49:10 +01:00
Ingo Weinhold
a1c42e7189 Replace missed alpha-raw release build profile occurrences 2013-11-23 16:58:53 +01:00
Ingo Weinhold
d85bab41bd Add target for building a Haiku package repository
* Add rule HaikuRepository to build a repository from a repository info
  file and a list of package files. It calls a build_haiku_repository
  script which does all the work.
* Add target <repository>haiku for building the Haiku package
repository.
  It should be built via "jam -q @alpha-raw build <repository>haiku";
  the build profile is only needed to activate all build features.
2013-11-23 01:46:04 +01:00
Ingo Weinhold
b2806f310e RepositoryRules: Automatic whitespace cleanup 2013-11-22 23:36:06 +01:00
Oliver Tappe
49b970d9c2 Reintroduce jam rule for creating remote repository for testing.
* 'jam build-remote-test-repository' can now be used to create a
  remote repository which will then be used by the build system,
  such that the resulting package set can be tested.
2013-11-14 23:48:56 +01:00
Oliver Tappe
e4bd712d11 Replace jam build-remote-repository with jam upload-packages
* We are moving the HaikuPorts repositories over from haiku-files.org
  to packages.haiku-os.org, and we will be creating new repositories
  during a push hook from now on. As a result, only a small helper for
  uploading new packages into the appropriate upload folder is required.
2013-11-03 23:53:31 +01:00
Ingo Weinhold
40b9ac787c Bootstrap image haikuports.config: Comment out PACKAGER
The developer building the packages should enter their own address
instead. I just noticed that a lot of the HaikuPorts package have a
packager attribute with the dummy address "The Haiku build system
<build-system@haiku-os.org>" due to the previous default value.
2013-10-10 23:19:48 +02:00
Jérôme Duval
4d07fb53d0 added a check for HOST_EXTENDED_REGEX_SED presence 2013-10-01 19:48:55 +02:00
François Revol
4d65f429d0 Fix bashisms
Besides, at least one test was probably incorrect,
trying to match /* with a single = in [[.
2013-09-29 03:01:39 +02:00
Oliver Tappe
af8587ce40 Added new jam command 'build-remote-repository'.
* build-remote-repository <packages> uploads the given set of packages
  to the server and builds the remote repository
2013-09-14 19:51:09 +02:00
Oliver Tappe
fb99605f49 Handle rigged source packages when determining port name.
* I'm not sure this makes much of a difference, but I thought it's 
  better to play it safe.
2013-09-02 12:03:06 +02:00
Ingo Weinhold
24796718b9 Bootstrap Haiku image: add source for more required packages
* Introduce build variable HAIKU_IMAGE_ADDITIONAL_PACKAGES which can be
  set to add additional packages to the image.
* Use HAIKU_IMAGE_ADDITIONAL_PACKAGES to support specifying a list of
  additional packages via the build-package-list build profile action.
* Add htmldoc and texi2html when building the list of the packages for
  which we need source packages when building the bootstrap Haiku image.
  We don't want them on the regular image, but they are needed to build
  some packages that are on it.
2013-08-17 09:20:35 +02:00
Ingo Weinhold
13f2fbbd9e Add build variable HAIKU_PORTER_CONCURRENT_JOBS
It can be set to the number of jobs specified on the haikuporter
command line. Speeds up the bootstrap build quite a bit.
2013-08-17 09:12:45 +02:00
Oliver Tappe
f81fbb0b59 Adjust to renamed option of haikuporter. 2013-08-07 13:42:16 +02:00
Ingo Weinhold
c965ee60b5 bootstrap image: build needed source packages and add them
* IsPackageAvailable, FetchPackage: Add flags parameter. The only
  flag supported ATM is nameResolved, indicating that the specified
  package name does not need to be resolved with respect to a secondary
  architecture anymore.
* Add build profile action "build-package-list". As an argument the
  file to which the list of all packages needed for the image is
  written. The rule BuildHaikuImagePackageList implements the action.
* BuildBootstrapRepositoryConfig: Does now require the variable
  HAIKU_REPOSITORY_TREE_PATH to be set on the config file target instead
  of hard-coding the path. Allows reuse of the actions.
* Add rules BuildHaikuPortsSourcePackageDirectory and
  BuildHaikuPortsRepositoryConfig. The former builds all HaikuPorts
  source packages needed to build the packages for an alpha image. The
  latter generates a haikuports.conf file for use on the bootstrap
  Haiku.
* HaikuImageBootstrap: Add directory /boot/home/haikuports which
  contains a subdirectory with the source packages and a
  haikuports.conf.
2013-08-06 23:02:10 +02:00
Oliver Tappe
0b171ae44a Fix two problems when building without secondary architecture. 2013-08-05 21:36:25 +02:00
Ingo Weinhold
93cfb72270 build features/bootstrap repo: support secondary arch
* Add rule FSplitPackageName. It splits a package name into port name
  and package suffix.
* FSetConditionsHold: Rename to FConditionsHold and replace the set
  parameter by a predicate rule parameter, thus adding more flexibility.
* FIsBuildFeatureEnabled: Use the faster check.
* Add rule FQualifiedBuildFeatureName. Given a build feature name, it
  prepends the current packaging architecture to yield a qualified
  feature name. Is used by the other build feature rules so that the
  same build feature can be configured differently for each arch.
* ExtractBuildFeatureArchives: The supplied list is now filtered via
  FFilterByBuildFeatures, allowing for build feature conditions in the
  list.
* Add rule InitArchitectureBuildFeatures. It is called early for each
  configured architecture, setting up some basic build features for it.
  "primary" is set for the primary architecture and a "secondary_<arch>"
  is set for each secondary architecture.
* BuildFeatures: Add secondary architecture support: Use the correct
  paths for libraries and headers (subdir for secondary architecture)
  and configure the icu and zlib sources only for the primary
  architecture.
* BootstrapPackageRepository: The package lists are now filtered via
  FFilterByBuildFeatures, allowing for build feature conditions in the
  lists.
* IsPackageAvailable, FetchPackage: Add secondary architecture support.
* HaikuPortsCross/x86_gcc2: Add icu and zlib x86 secondary packages.

The second stage Haiku cross devel package for the secondary
architecture can now be built.
2013-08-05 07:09:45 +02:00
Ingo Weinhold
b0944c78b0 More work towards hybrid support
* All packaging architecture dependent variables do now have a
  respective suffix and are set up for each configured packaging
  architecture, save for the kernel and boot loader variables, which
  are still only set up for the primary architecture.
  For convenience TARGET_PACKAGING_ARCH, TARGET_ARCH, TARGET_LIBSUPC++,
  and TARGET_LIBSTDC++ are set to the respective values for the primary
  packaging architecture by default.
* Introduce a set of MultiArch* rules to help with building targets for
  multiple packaging architectures. Generally the respective targets are
  (additionally) gristed with the packaging architecture. For libraries
  the additional grist is usually omitted for the primary architecture
  (e.g. libroot.so and <x86>libroot.so for x86_gcc2/x86 hybrid), so that
  Jamfiles for targets built only for the primary architecture don't
  need to be changed.
* Add multi-arch build support for all targets needed for the stage 1
  cross devel package as well as for libbe (untested).
2013-08-01 08:54:06 +02:00
Ingo Weinhold
7b2174a27d BootstrapPackageRepository: serialize package creation
Since haikuporter uses a common directory for building the ports, we
can't build them concurrently. This makes jamming with multiple jobs
safe again.
2013-07-21 19:39:40 +02:00
Ingo Weinhold
e173a1ec1c Integrate building the HaikuPorts bootstrap packages
* Add configure option --bootstrap which allows specifying the
  haikuporter and HaikuPorts repository paths.
* Add rules for supporting a second repository type. The
  PackageRepository rule is now private and RemotePackageRepository is
  used for remote repositories. The new BootstrapPackageRepository rule
  is for defining a bootstrap repository (there will probably be only
  the HaikuPorts cross repository) whose packages can be built as needed
  via haikuporter.
* Rename DownloadPackage to FetchPackage.
* Define repository HaikuPortsCross.
* HaikuCrossDevel package(s): There are now two sets of packages: A
  "stage1" set with the same content as before and a final set
  additionally containing the libraries libbe, libnetwork, libpackage.
  Those are needed for building the libsolv bootstrap package while for
  building them we need other bootstrap packages (ICU, libz).

This is basically all that's required to build a bootstrap Haiku
completely from sources, with a few caveats:
* There's no ICU bootstrap recipe yet (so one has to cheat and use the
  prebuilt package ATM).
* Probably doesn't work on Haiku yet (tested on Linux only).
* A 32 bit environment must be used (otherwise building the gcc 2
  bootstrap package fails).
* Building with multiple jobs doesn't work yet, since haikuporter uses
  common directories for building different packages and there's no
  explicit serialization yet.
* Haven't tested the resulting image save for booting it. So it probably
  needs a bit more work before it can actually build the final
  HaikuPorts packages.
2013-07-21 04:10:48 +02:00
Ingo Weinhold
38f07871ce rule AddRepositoryPackages: add debugInfoPackage parameter
It works just like the sourcePackages parameter.
2013-07-12 13:27:56 +02:00
Ingo Weinhold
014eed80e2 Change repository directory layout
* Under the base URL there are supposed to be the repository files and a
  subdirectory "packages".
* Fix the repository URL related confusion introduced earlier. The URL
in
  the repository info (and thus in the repository file) is supposed to
  be the base URL for the repository. It is not a (potentially)
  different base URL for the package files. Package and repository
  files were supposed to live in the same directory. Now, by requiring
  the package files to live in a subdirectory -- which can also be a
  symlink -- we gain some flexibility.
  The URL in the repository config is usually the same as the in the
  repository info, unless it refers to a mirror site. This allows for
  mirrors to copy the original repository verbatim.
* Remove the PackageURL rule and introduce a DownloadPackage rule
  instead. The URL for a package file cannot be computed in the jam
  parsing phase anymore, as it contains the hash value of the package
  list.
* BRepositoryConfig: Add PackagesURL() for convenience.
2013-07-06 18:06:02 +02:00
Ingo Weinhold
d72ce06673 Adjust to changed HaikuPorts repository layout
The package kit actually requires the files "repo", "repo.info",
"repo.sha256" to be located under the repository base URL, so the
approach to name the repository file "repo-<hash>" doesn't work.
Now there's a directory "<hash>" which contains the files.

This commit moves the computation of the hash and downloading the
repository file from the build_haiku_image script to the jam build
system. The repo.info is also downloaded and a repository config file
is generated.
2013-07-06 02:29:51 +02:00
Ingo Weinhold
627626afd4 GeneratedRepositoryPackageList: defeat localization
Set LC_ALL=C, so "sort" produces useful results.
2013-07-05 14:28:09 +02:00
Ingo Weinhold
98c6dfa41e Switch build system from optional package to repositories
* Build libsolv and the dependency solver part of the package kit for
  the build platform.
* Add build tool get_package_dependencies. Given a list of package files
  and a list of repository files it determines the additional packages
  that need to be retrieved from the repositories and prints their URLs.
* Add rules to work with external repositories in the build system
  (build/jam/RepositoryRules):
  - PackageRepository declares an external repository with all its
    packages. The URL of the repository file isn't specified. It is
    computed from a given base URL and the SHA256 hash of the list of
    package files.
  - GeneratedRepositoryPackageList generates a file containing the file
    names of all packages in a repository.
  - IsPackageAvailable returns whether a package is available in any
    repository.
  - PackageURL returns the URL for a package.
* Declare the HaikuPorts repository for x86_gcc2
  (build/jam/repositories/HaikuPorts/x86_gcc2).
* Add rule AddHaikuImagePackages to add a package to the image and rule
  IsHaikuImagePackageAdded to determine whether a package has been
  added.
* OptionalPackages: Remove all entries that just downloaded and
  installed an external package. AddHaikuImagePackages can be used
  instead and is used in the remaining entries. Also move the remaining
  optional package dependency declarations from
  OptionalPackageDependencies here.
* ExtractBuildFeatureArchives: Instead of the URL parameter a package
  name must be specified now. This allows to simplify BuildFeatures
  significantly, since there's no dealing with URLs anymore. "if" out
  the entries that aren't supported yet.
* build_haiku_image: For the packages installed in system and common
  resolve their dependencies and download and install them as well.
2013-07-05 10:51:42 +02:00