Aren/haiku
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
a5061ecec5
haiku/docs/develop/midi/design.rst

538 lines
25 KiB
ReStructuredText
Raw Normal View History

Generate developer docs with Sphinx An effort was started some time ago to consolidate all internal documentation in the git tree. However, this was just an accumulation of files in various formats without any strucutre or way to browse it, which results in no one even knowing that we have docs here. This converts most of the files to restructuredtext and uses Sphinx to generate an HTML browsable user manual (with a table of content and a first attempt to put things in a global hierarchy). There are almost no changes to the documentation content in this commit (some obviously obsolete things were removed). The plan is to get the toolchain up and running to make these docs easily available, and only then see about improving the content. We can migrate some things off the wiki and website, and rework the table of contents to have some more hierarchy levels because currently it's a bit messy. Change-Id: I924ac9dc6e753887ab56f18a09bdb0a1e1793bfd Reviewed-on: https://review.haiku-os.org/c/haiku/+/4370 Reviewed-by: Niels Sascha Reedijk <niels.reedijk@gmail.com>
2021-08-24 18:45:36 +03:00
Midi Kit design
===============
The Midi Kit consists of the midi_server and two shared libraries,
libmidi2.so and libmidi.so. The latter is the "old" pre-R5 Midi Kit and
has been re-implemented using the facilities from libmidi2, which makes
it fully compatible with the new kit. This document describes the design
and implementation of the OpenBeOS midi_server and libmidi2.so.
The midi_server has two jobs: it keeps track of the endpoints that the
client apps have created, and it publishes endpoints for the devices
from /dev/midi. (This last task could have been done by any other app,
but it was just as convenient to make the midi_server do that.) The
libmidi2.so library also has two jobs: it assists the midi_server with
the housekeeping stuff, and it allows endpoints to send and receive MIDI
events. (That's right, the midi_server has nothing to do with the actual
MIDI data.)
--------------
Ooh, pictures
-------------
The following image shows the center of Midi Kit activity, the
midi_server, and its data structures:
|image0|
And here is the picture for libmidi2.so:
|image1|
Note that these diagrams give only a conceptual overview of who is
responsible for which bits of data. The actual implementation details of
the kit may differ.
--------------
Housekeeping
------------
- The design for our implementation of the midi2 "housekeeping"
protocol roughly follows `what Be did <oldprotocol.html>`__, although
there are some differences. In Be's implementation, the BMidiRosters
only have BMidiEndpoints for remote endpoints if they are registered.
In our implementation, the BMidiRosters have BMidiEndpoint objects
for *all* endpoints, including remote endpoints that aren't published
at all. If there are many unpublished endpoints in the system, our
approach is less optimal. However, it made the implementation of the
Midi Kit much easier ;-)
- Be's libmidi2.so exports the symbols "midi_debug_level" and
"midi_dispatcher_priority", both int32's. Our libmidi2 does not use
either of these. But even though these symbols are not present in the
headers, some apps may use them nonetheless. That's why our libmidi2
exports those symbols as well.
- The name of the message fields in Be's implementation of the protocol
had the "be:" prefix. Our fields have a "midi:" prefix instead.
Except for the fields in the B_MIDI_EVENT notification messages,
because that would break compatibility with existing apps.
Initialization
~~~~~~~~~~~~~~
- The first time an app uses a midi2 class, the
BMidiRoster::MidiRoster() method sends an 'Mapp' message to the
midi_server, and blocks (on a semaphore). This message includes a
messenger to the app's BMidiRosterLooper object. The server adds the
app to its list of registered apps. Then the server asynchronously
sends back a series of 'mNEW' message notifications for all endpoints
on the roster, and 'mCON' messages for all existing connections. The
BMidiRosterLooper creates BMidiEndpoint objects for these endpoints
and adds them to its local roster; if the app is watching, it also
sends out corresponding B_MIDI_EVENT notifications. Finally, the
midi_server sends an 'mAPP' message to notify the app that it has
been successfully registered. Upon receipt, BMidiRoster::MidiRoster()
unblocks and returns control to the client code. This handshake is
the only asynchronous message exchange; all the other requests have a
synchronous reply.
- If the server detects an error during any of this (incorrect message
format, delivery failure, etc.) it simply ignores the request and
does not try to send anything back to the client (which is most
likely impossible anyway). If the app detects an error (server sends
back meaningless info, cannot connect to server), it pretends that
everything is hunkey dorey. (The API has no way of letting the client
know that the initialization succeeded.) Next time the app tries
something, the server either still does not respond, or it ignores
the request (because this app isn't properly registered). However, if
the app does not receive the 'mAPP' message, it will not unblock, and
remains frozen for all eternity.
- BMidiRoster's MidiRoster() method creates the one and only
BMidiRoster instance on the heap the first time it is called. This
instance is automatically destroyed when the app quits.
Error handling
~~~~~~~~~~~~~~
- If some error occurs, then the reply message is only guaranteed to
contain the "midi:result" field with some non- zero error code.
libmidi2 can only assume that the reply contains other data on
success (i.e. when "midi:result" is B_OK).
- The timeout for delivering and responding to a message is about 2
seconds. If the client receives no reply within that time, it assumes
the request failed. If the server cannot deliver a message within 2
seconds, it assumes the client is dead and removes it (and its
endpoints) from the roster. Of course, these assumptions may be
false. If the client wasn't dead and tries to send another request to
the server, then the server will now ignore it, since the client app
is no longer registered.
- Because we work with timeouts, we must be careful to avoid
misunderstandings between the midi_server and the client app. Both
sides must recognize the timeout, so they both can ignore the
operation. If, however, the server thinks that everything went okay,
but the client flags an error, then the server and the client will
have two different ideas of the current state of the roster. Of
course, those situations must be avoided.
- Although apps register themselves with the midi_server, there is no
corresponding "unregister" message. The only way the server
recognizes that an app and its endpoints are no longer available is
when it fails to deliver a message to that app. In that case, we
remove the app and all its endpoints from the roster. To do this, the
server sends "purge endpoint" messages to itself for all of the app's
endpoints. This means we don't immediately throw the app away, but we
schedule that for some time in the future. That makes the whole event
handling mechanism much cleaner. There is no reply to the purge
request. (Actually, we *do* immediately throw away the app_t object,
since that doesn't really interfere with anything.) (If there are
other events pending in the queue which also cause notifications,
then the server may send multiple purge messages for the same
endpoints. That's no biggie, because a purge message will be ignored
if its endpoint no longer exists.)
- As mentioned above, the midi_server ignores messages that do not come
from a registered app, although it does send back an error reply. In
the case of the "purge endpoint" message, the server makes sure the
message was local (i.e. sent by the midi_server itself).
- Note: BMessage's SendReply() apparently succeeds even if you kill the
app that the reply is intended for. This is rather strange, and it
means that you can't test delivery error handling for replies by
killing the app. (You *can* kill the app for testing the error
handling on notifications, however.)
Creating and deleting endpoints
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- When client code creates a new BMidiLocalProducer or
BMidiLocalConsumer endpoint, we send an 'Mnew' message to the server.
Unlike Be's implementation, the "name" field is always present, even
if the name is empty. After adding the endpoint to the roster, the
server sends 'mNEW' notifications to all other applications. Upon
receipt of this notification, the BMidiRosterLoopers of these apps
create a new BMidiEndpoint for the endpoint and add it to their
internal list of endpoints. The app that made the request receives a
reply with a single "midi:result" field.
- When you "new" an endpoint, its refcount is 1, even if the creation
failed. (For example, if the midi_server does not run.) When you
Acquire(), the refcount is bumped. When you Release(), it is
decremented. When refcount drops to 0, the endpoint object "deletes"
itself. (So client code should never use an endpoint after having
Release()'d it, because the object may have just been killed.) When
creation succeeds, IsValid() returns true and ID() returns a valid ID
(> 0). Upon failure, IsValid() is false and ID() returns 0.
- After the last Release() of a local endpoint, we send 'Mdel' to let
the midi_server know the endpoint is now deleted. We don't expect a
reply back. If something goes wrong, the endpoint is deleted
regardless. We do not send separate "unregistered" notifications,
because deleting an endpoint implies that it is removed from the
roster. For the same reason, we also don't send separate
"disconnected" notifications.
- The 'mDEL' notification triggers a BMidiRosterLooper to remove the
corresponding BMidiEndpoint from its internal list. This object is
always a proxy for a remote endpoint. The remote endpoint is gone,
but whether we can also delete the proxy depends on its reference
count. If no one is still using the object, its refcount is zero, and
we can safely delete the object. Otherwise, we must defer destruction
until the client Release()'s the object.
- If you "delete" an endpoint, your app drops into the debugger.
- If you Release() an endpoint too many times, your app *could* drop
into the debugger. It might also crash, because you are now using a
dead object. It depends on whether the memory that was previously
occupied by your endpoint object was overwritten in the mean time.
- You are allowed to pass NULL into the constructors of
BMidiLocalConsumer and BMidiLocalProducer, in which case the
endpoint's name is simply an empty string.
Changing endpoint attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- An endpoint can be "invalid". In the case of a proxy this means that
the remote endpoint is unregistered or even deleted. Local endpoints
can only be invalid if something went wrong during their creation (no
connection to server, for example). You can get the attributes of
invalid objects, but you cannot set them. Any attempts to do so will
return an error code.
- For changing the name, latency, or properties of an endpoint,
libmidi2 sends an 'Mchg' message with the fields that should be
changed, "midi:name", "midi:latency", or "midi:properties".
Registering or unregistering an endpoint also sends such an 'Mchg'
message, because we consider the "registered" state also an
attribute, in "midi:registered". The message obviously also includes
the ID of the endpoint in question. Properties are sent using a
different message, because the properties are not stored inside the
BMidiEndpoints.
- After handling the 'Mchg' request, the midi_server broadcasts an
'mCHG' notification to all the other apps. This message has the same
contents as the original request.
- If the 'Mchg' message contains an invalid "midi:id" (i.e. no such
endpoint exists or it does not belong to the app that sent the
request), the midi_server returns an error code, and it does not
notify the other apps.
- If you try to Register() an endpoint that is already registered,
libmidi2 does not send a message to the midi_server but simply
returns B_OK. (Be's implementation *did* send a message, but our
libmidi2 also keeps track whether an endpoint is registered or not.)
Although registering an endpoint more than once doesn't make much
sense, it is not considered an error. Likewise for Unregister()ing an
endpoint that is not registered.
- If you try to Register() or Unregister() a remote endpoint, libmidi2
immediately returns an error code, and does not send a message to the
server. Likewise for a local endpoints that are invalid (i.e. whose
IsValid() function returns false).
- BMidiRoster::Register() and Unregister() do the same thing as
BMidiEndpoint::Register() and Unregister(). If you pass NULL into
these functions, they return B_BAD_VALUE.
- SetName() ignores NULL names. When you call it on a remote endpoint,
SetName() does nothing. SetName() does not send a message if the new
name is the same as the current name.
- SetLatency() ignores negative values. SetLatency() does not send a
message if the new latency is the same as the current latency. (Since
SetLatency() lives in BMidiLocalConsumer, you can never use it on
remote endpoints.)
- We store a copy of the endpoint properties in each BMidiEndpoint. The
properties of new endpoints are empty. GetProperties() copies this
BMessage into the client's BMessage. GetProperties() returns NULL if
the message parameter is NULL.
- SetProperties() returns NULL if the message parameter is NULL. It
returns an error code if the endpoint is remote or invalid.
SetProperties() does *not* compare the contents of the new BMessage
to the old, so it will always send out the change request.
Connections
~~~~~~~~~~~
- BMidiProducer::Connect() sends an 'Mcon' request to the midi_server.
This request contains the IDs of the producer and the consumer you
want to connect. The server sends back a reply with a result code. If
it is possible to make this connection, the server broadcasts an
'mCON' notification to all other apps. In one of these apps the
producer is local, so that app's libmidi2 calls the
BMidiLocalProducer::Connected() hook.
- You are not allowed to connect the same producer and consumer more
than once. The midi_server checks for this. It also returns an error
code if you try to disconnect two endpoints that were not connected.
- Disconnect() sends an 'Mdis' request to the server, which contains
the IDs of the producer and consumer that you want to disconnect. The
server replies with a result code. If the connection could be broken,
it also sends an 'mDIS' notification to the other apps. libmidi2
calls the local producer's BMidiLocalProducer::Disconnected() hook.
- Connect() and Disconnect() immediately return an error code if you
pass a NULL argument, or if the producer or consumer is invalid.
- When you Release() a local consumer that is connected, all apps will
go through their producers, and throw away this consumer from their
connection lists. If one of these producers is local, we call its
Disconnected() hook. If you release a local producer, this is not
necessary.
Watching
~~~~~~~~
- When you call StartWatching(), the BMidiRosterLooper remembers the
BMessenger, and sends it B_MIDI_EVENT notifications for all
registered remote endpoints, and the current connections between
them. It does not let you know about local endpoints. When you call
StartWatching() a second time with the same BMessenger, you'll
receive the whole bunch of notifications again. StartWatching(NULL)
is not allowed, and will be ignored (so it is not the same as
StopWatching()).
Thread safety
~~~~~~~~~~~~~
- Within libmidi2 there are several possible race conditions, because
we are dealing with two threads: the one from BMidiRosterLooper and a
thread from the client app, most likely the BApplication's main
thread. Both can access the same data: BMidiEndpoint objects. To
synchronize these threads, we lock the BMidiRosterLooper, which is a
normal BLooper. Anything happening in BMidiRosterLooper's message
handlers is safe, because BLoopers are automatically locked when
handling a message. Any other operations (which run from a different
thread) must first lock the looper if they access the list of
endpoints or certain BMidiEndpoint attributes (name, properties,
etc).
- What if you obtain a BMidiEndpoint object from FindEndpoint() and at
the same time the BMidiRosterLooper receives an 'mDEL' request to
delete that endpoint? FindEndpoint() locks the looper, and bumps the
endpoint object before giving it to you. Now the looper sees that the
endpoint's refcount is larger than 0, so it won't delete it (although
it will remove the endpoint from its internal list). What if you
Acquire() or Release() a remote endpoint while it is being deleted by
the looper? That also won't happen, because if you have a pointer to
that endpoint, its refcount is at least 1 and the looper won't delete
it.
- It is not safe to use a BMidiEndpoint and/or the BMidiRoster from
more than one client thread at a time; if you want to do that, you
should synchronize access to these objects yourself. The only
exception is the Spray() functions from BMidiLocalProducer, since
most producers have a separate thread to spray their MIDI events.
This is fine, as long as that thread isn't used for anything else,
and it is the only one that does the spraying.
- BMidiProducer objects keep a list of consumers they are connected to.
This list can be accessed by several threads at a time: the client's
thread, the BMidiRosterLooper thread, and possibly a separate thread
that is spraying MIDI events. We could have locked the producer using
BMidiRosterLooper's lock, but that would freeze everything else while
the producer is spraying events. Conversely, it would freeze all
producers while the looper is talking to the midi_server. To lock
with a finer granularity, each BMidiProducer has its own BLocker,
which is used only to lock the list of connected consumers.
Misc remarks
~~~~~~~~~~~~
- BMidiEndpoint keeps track of its local/remote state with an "isLocal"
variable, and whether it is a producer/consumer with "isConsumer". It
also has an "isRegistered" field to remember whether this endpoint is
registered or not. Why not lump all these different states together
into one "flags" bitmask? The reason is that isLocal only makes sense
to this application, not to others. Also, the values of isLocal and
isConsumer never change, but isRegistered does. It made more sense
(and clearer code) to separate them out. Finally, isRegistered does
not need to be protected by a lock, even though it can be accessed by
multiple threads at a time. Reading and writing a bool is atomic, so
this can't get messed up.
The messages
~~~~~~~~~~~~
::
Message: Mapp (MSG_REGISTER_APP)
BMessenger midi:messenger
Reply:
(no reply)
Message: mAPP (MSG_APP_REGISTERED)
(no fields)
Message: Mnew (MSG_CREATE_ENDPOINT)
bool midi:consumer
bool midi:registered
char[] midi:name
BMessage midi:properties
int32 midi:port (consumer only)
int64 midi:latency (consumer only)
Reply:
int32 midi:result
int32 midi:id
Message: mNEW (MSG_ENPOINT_CREATED)
int32 midi:id
bool midi:consumer
bool midi:registered
char[] midi:name
BMessage midi:properties
int32 midi:port (consumer only)
int64 midi:latency (consumer only)
Message: Mdel (MSG_DELETE_ENDPOINT)
int32 midi:id
Reply:
(no reply)
Message: Mdie (MSG_PURGE_ENDPOINT)
int32 midi:id
Reply:
(no reply)
Message: mDEL (MSG_ENDPOINT_DELETED)
int32 midi:id
Message: Mchg (MSG_CHANGE_ENDPOINT)
int32 midi:id
int32 midi:registered (optional)
char[] midi:name (optional)
int64 midi:latency (optional)
BMessage midi:properties (optional)
Reply:
int32 midi:result
Message: mCHG (MSG_ENDPOINT_CHANGED)
int32 midi:id
int32 midi:registered (optional)
char[] midi:name (optional)
int64 midi:latency (optional)
BMessage midi:properties (optional)
--------------
MIDI events
-----------
- MIDI events are always sent from a BMidiLocalProducer to a
BMidiLocalConsumer. Proxy endpoint objects have nothing to do with
this. During its construction, the local consumer creates a kernel
port. The ID of this port is published, so everyone knows what it is.
When a producer sprays an event, it creates a message that it sends
to the ports of all connected consumers.
- This means that the Midi Kit considers MIDI messages as discrete
events. Hardware drivers chop the stream of incoming MIDI data into
separate events that they send out to one or more kernel ports.
Consumers never have to worry about parsing a stream of MIDI data,
just about handling a bunch of separate events.
- Each BMidiLocalConsumer has a (realtime priority) thread associated
with it that waits for data to arrive at the port. As soon as a new
MIDI message comes in, the thread examines it and feeds it to the
Data() hook. The Data() hook ignores the message if the "atomic" flag
is false, or passes it on to one of the other hook functions
otherwise. Incoming messages are also ignored if their contents are
not valid; for example, if they have too few or too many bytes for a
certain type of MIDI event.
- Unlike the consumer, BMidiLocalProducer has no thread of its own. As
a result, spraying MIDI events always happens in the thread of the
caller. Because the consumer port's queue is only 1 message deep,
spray functions will block if the consumer thread is already busy
handling another MIDI event. (For this reason, the Midi Kit does not
support interleaving of real time messages with lower priority
messages such as sysex dumps, except at the driver level.)
- The producer does not just send MIDI event data to the consumer, it
also sends a 20-byte header describing the event. The total message
looks like this:
+---------+------------------------------+
| 4 bytes | ID of the producer |
+---------+------------------------------+
| 4 bytes | ID of the consumer |
+---------+------------------------------+
| 8 bytes | performance time |
+---------+------------------------------+
| 1 byte | atomic (1 = true, 0 = false) |
+---------+------------------------------+
| 3 bytes | padding (0) |
+---------+------------------------------+
| x bytes | MIDI event data |
+---------+------------------------------+
- In the case of a sysex event, the SystemExclusive() hook is only
called if the first byte of the message is 0xF0. The sysex end marker
(0xF7) is optional; only if the last byte is 0xF7 we strip it off.
This is unlike Be's implementation, which all always strips the last
byte even when it is not 0xF7. According to the MIDI spec, 0xF7 is
not really required; any non-realtime status byte ends a sysex
message.
- SprayTempoChange() sends 0xFF5103tttttt, where tttttt is
60,000,000/bpm. This feature is not really part of the MIDI spec, but
an extension from the SMF (Standard MIDI File) format. Of course, the
TempoChange() hook is called in response to this message.
- The MIDI spec allows for a number of shortcuts. A Note On event with
velocity 0 is supposed to be interpreted as a Note Off, for example.
The Midi Kit does not concern itself with these shortcuts. In this
case, it still calls the NoteOn() hook with a velocity parameter of
0.
- The purpose of BMidiLocalConsumer's AllNotesOff() function is not
entirely clear. All Notes Off is a so-called "channel mode message"
and is generated by doing a SprayControlChange(channel,
B_ALL_NOTES_OFF, 0). BMidi has an AllNotesOff() function that sends
an All Notes Off event to all channels, and possible Note Off events
to all keys on all channels as well. I suspect someone at Be was
confused by AllNotesOff() being declared "virtual", and thought it
was a hook function. Only that would explain it being in
BMidiLocalConsumer as opposed to BMidiLocalProducer, where it would
have made sense. The disassembly for Be's libmidi2.so shows that
AllNotesOff() is empty, so to cut a long story short, our
AllNotesOff() simply does nothing and is never invoked either.
- There are several types of System Common events, each of which takes
a different number of data bytes (0, 1, or 2). But
SpraySystemCommon() and the SystemCommon() hook are always given 2
data parameters. The Midi Kit simply ignores the extra data bytes; in
fact, in our implementation it doesn't even send them. (The Be
implementation always sends 2 data bytes, but that will confuse the
Midi Kit if the client does a SprayData() of a common event instead.
In our case, that will still invoke the SystemCommon() hook, because
we are not as easily fooled.)
- Handling of timeouts is fairly straightforward. When reading from the
port, we specify an absolute timeout. When the port function returns
with a B_TIMED_OUT error code, we call the Timeout() hook. Then we
reset the timeout value to -1, which means that timeouts are disabled
(until the client calls SetTimeout() again). This design means that a
call to SetTimeout() only takes effect the next time we read from the
port, i.e. after at least one new MIDI event is received (or the
previous timeout is triggered). Even though BMidiLocalConsumer's
timeout and timeoutData values are accessed by two different threads,
I did not bother to protect this. Both values are int32's and
reading/writing them should be an atomic operation on most processors
anyway.
.. |image0| image:: midi_server.png
.. |image1| image:: libmidi2.png
Reference in New Issue Copy Permalink