a5061ecec5
An effort was started some time ago to consolidate all internal documentation in the git tree. However, this was just an accumulation of files in various formats without any strucutre or way to browse it, which results in no one even knowing that we have docs here. This converts most of the files to restructuredtext and uses Sphinx to generate an HTML browsable user manual (with a table of content and a first attempt to put things in a global hierarchy). There are almost no changes to the documentation content in this commit (some obviously obsolete things were removed). The plan is to get the toolchain up and running to make these docs easily available, and only then see about improving the content. We can migrate some things off the wiki and website, and rework the table of contents to have some more hierarchy levels because currently it's a bit messy. Change-Id: I924ac9dc6e753887ab56f18a09bdb0a1e1793bfd Reviewed-on: https://review.haiku-os.org/c/haiku/+/4370 Reviewed-by: Niels Sascha Reedijk <niels.reedijk@gmail.com>
121 lines
4.7 KiB
ReStructuredText
121 lines
4.7 KiB
ReStructuredText
HaikuDepot and Server Interactions
|
||
==================================
|
||
|
||
Introduction
|
||
------------
|
||
|
||
This document aims to outline the general approach taken within the
|
||
HaikuDepot application with regard to coordinating processes that relate
|
||
to fetching and consuming data from remote systems.
|
||
|
||
There are two main sources of remote data that are downloaded and
|
||
consumed from network sources into the HaikuDepot desktop application;
|
||
|
||
- Repository HPKR data from a Haiku mirror such as “HaikuPorts”
|
||
- Meta-data related to packages from
|
||
`HaikuDepotServer <http://depot.haiku-os.org>`__ (HDS) such as icons,
|
||
localizations, ratings and so on.
|
||
|
||
Process, ProcessNode and Coordinator
|
||
------------------------------------
|
||
|
||
A *Process* (root class ``AbstractProcess``) is a class that takes
|
||
responsibility for some aspect of pulling material down from a network
|
||
source and processing it.
|
||
|
||
A *ProcessNode* is a holder for a Process, but also takes responsibility
|
||
for the following;
|
||
|
||
- Maintaining the relationship between the Processes. For example, if
|
||
Process A needs to complete before Process B then the ProcessNode
|
||
would record this fact. It does this by storing *predecessor* and
|
||
*successor* ProcessNodes.
|
||
- Starting the held Process in a newly spawned thread.
|
||
- Stopping the held Process.
|
||
|
||
A *Coordinator* holds a list of ProcessNodes. It will start, stop and
|
||
cancel nodes as necessary such that, in an ideal case, the various
|
||
ProcessNodes are completed in the correct order.
|
||
|
||
The *ProcessCoordinatorFactory* is able to create Coordinators.
|
||
|
||
Bulk Load Processes
|
||
-------------------
|
||
|
||
The following diagram shows the logical dependencies of the various
|
||
Processes that are involved in refreshing the HPKR data from remote
|
||
repositories and then loading data from the HDS system.
|
||
|
||
.. figure:: images/processes.svg
|
||
:alt: Process Dependencies
|
||
|
||
Process Dependencies
|
||
|
||
For example, the ``ServerRepositoryDataUpdateProcess`` must wait until
|
||
the ``LocalRepositoryUpdateProcess`` has completed before it is able to
|
||
be started. It is the reponsibility of the Coordinator to ensure that
|
||
this sequencing is enforced. There are many instances of
|
||
``ServerPkgDataUpdateProcess`` shown because there will be one launched
|
||
for each of the Repositories for which data will be downloaded;
|
||
“HaikuDepot” etc…
|
||
|
||
Process / ProcessNode / Coordinator
|
||
-----------------------------------
|
||
|
||
The following diagram shows the relationship and interplay between the
|
||
various objects that are involved in running a larger task. Only
|
||
fictional Processes are shown to keep the diagram tidy. See above for
|
||
the actual Processes.
|
||
|
||
.. figure:: images/process-interplay.svg
|
||
:alt: Process Relationship and Interplay
|
||
|
||
Process Relationship and Interplay
|
||
|
||
Dotted lines show associations between elements and red lines show
|
||
interaction or data-flow. Green arrows here demonstratively show some
|
||
dependency; Process C cannot start until A and B are completed.
|
||
|
||
The MainWindow owns the Coordinator for the life-span of undertaking
|
||
some larger task.
|
||
|
||
Each Process is coupled with a ProcessNode and then the Coordinator has
|
||
a list of the ProcessNodes-s. The Processes are generally writing to the
|
||
local disk system (often with compressed files) to cache data (see
|
||
``~/config/cache/HaikuDepot``) and also relay data into the ``Model``
|
||
object that maintains state for the HaikuDepot desktop application.
|
||
|
||
The Processes communicate when they have finished to the Coordinator and
|
||
it is at these events that the Coordinator is able to introspect the
|
||
state of the Processes in order to know what to do next.
|
||
|
||
The Coordinator also communicates with MainWindow. It communicates with
|
||
the MainWindow in order to signal changes or progress in the overall
|
||
larger task. The MainWindow also uses these events to discover when the
|
||
Coordinator has completely finished.
|
||
|
||
Failure
|
||
-------
|
||
|
||
A Process may fail or be stopped. If a Process fails or is stopped then
|
||
successor Processes, or those that would have run after the failed
|
||
process, are stopped so that they will not run.
|
||
|
||
The Coordinator will still try to complete any other Processes that
|
||
could still run or are running already.
|
||
|
||
Upon the Coordinator completing, the Coordinator will signal to the
|
||
MainWindow client the change in state and then the MainWindow will be
|
||
able to identify that the Coordinator has completed, but that something
|
||
has gone wrong along the way.
|
||
|
||
Concurrency
|
||
-----------
|
||
|
||
It is important to note that Processes may run concurrently. The
|
||
Processes’ are modelled by the Coordinator as a list rather than a tree.
|
||
The dependencies are likely to form a tree or web of Processes that
|
||
dictates the order of execution, but it is also quite possible to have
|
||
multiple non-intersecting trees or webs such that Processes will execute
|
||
independently.
|