The surface's damage region is not relative to the output but
lives in its own universe starting at origin 0,0.
Also fixed the drawing coordinates used in the XRENDER code path
which is used with /smart-sizing
Another issue revealed during testing is that older Windows systems
cannot handle the reserved file names well. While Windows 8 and 10 are
fine (they silently abort the file transfer), using reserved names with
Windows 7 can flat out crash explorer.exe or result into weird error
messages like "fatal error: 0x00000000 ERROR_SUCCESS".
This is not required by MS-RDPECLIP specification, but we should try to
avoid this issue as not using reserved file names seems to be assumed
a common sense in Windows protocols.
The most convenient way to handle the issue would be on wClipboard level
so that WinPR's clients do not bother with it. We should prohibit the
reserved names from being used in FILEDESCRIPTOR, failing the conversion
if we see such a file.
POSIX subsystem (the only one at the moment) handles remote file names
in two places so move the Unicode conversion and the new validation
check into a separate function.
The reserved file name predicate is placed into <winpr/file.h> so that
it can be used in other places too. For example, other wClipboard local
file subsystems will need it. (It would be really nice to enforce this
check somewhere in the common code, so that the subsystems can't miss
it, but other places can miss some errors thus we're doing it here, as
early as possible.)
The predicate acts on separate file name components rather than full
file names because the backslash is a reserved character too. If we
process full file names this can result in phantom directory entry in
the remote file name. Not to say that handling ready-made components
spares us from splitting the full file name to extract them :)
The implementation is... a bit verbose, but that's fine by me. In the
absence of functions for case-insensitive wide string comparison and
the need to check for the [0-9] at the end of some file names this is
quite readable. Thanks to FAT and NTFS for being case-insensitive and
to MS-DOS for having reserved file names in the first place.
The file contents PDUs support 64-bit file sizes and offsets, but
MS-RDPECLIP explicitly says in 2.2.5.3 File Contents Request PDU that
file larger that 4 gigabytes are not supported by the server. It turns
out that the supported size is even lower than that. The server cannot
correctly handle files larger than 2 gigabytes (inclusive). When faced
with such files it correctly retireves the lower part, but fails to
accept any data past that boundary. After receiving a file range reply
the server repeats the file range request with the same offset, and
again, and again, and again, making no progress and blocking the file
transfer indefinitely. This is not the behavior we would like to have.
Microsoft support site acknowledges and documents the issue [1],
suggesting the users to use disk drive redirection instead to transfer
large files. (File transfers via cliprdr are considerably slower than
disk drive redirection so the suggestion makes very much sense.)
However, we would like to avoid the lockdown of the remote session if
the user does attempt to transfer such files so we add a size check.
Putting it into the conversion from FILEDESCRIPTOR to CLIPRDR_FILELIST
is not an ideal place (the clients may not use the common utilities),
but that's good enough currently.
[1]: https://support.microsoft.com/en-us/help/2258090
Do not try to 'helpfully' fixup the length of the data provided by the
client when FILECONTENTS_SIZE is present. This can lead to a crash if
the client wants to report an error in msgFlags, sets cbRequested to
zero, and does not provide any data in requestedData. For example,
XFreeRDP does this in xf_cliprdr_send_file_contents_failure() and
xf_cliprdr_clipboard_file_size_failure().
clipDataId is an optional field of CLIPRDR_FILECONTENTS_REQUEST.
The client should not send it to the server without sending a prior
CLIPRDR_LOCK_CLIPDATA request. The reverse is true as well: the
server should not include these additional 4 bytes without locking
the file in question.
The value zero is a valid ID, it cannot be used as a sentinel value.
Introduce a separate flag to tell whether the clipDataId has been set
and can be relied upon.
Also fix formatting. These stupid line breaks have negative impact on
readability, and the lines do fit into the 100 column limit either way.
Now that we've got everything in place to handle files it's time to tell
the server that we can actually do this.
MS-RDPECLIP 3.2.5.1.3 Sending a Client Clipboard Capabilities PDU asks
us politely to not advertise file clipping support if the server did not
do that itself. Thus we need to parse the capabilities sent by the
server and take a note whether it supports file clipping.
There is also no point in advertising file clipping support if
wClipboard failed to initalize any local file subsystem, in which case
we cannot handle files for real. Take a note of this as well when we
register the file formats.
If everthing is really in place and the stars shine upon us then we are
allowed to set CB_STREAM_FILECLIP_ENABLED in the capabilities. There is
no command line switch to disable file clipping (and there is little
reason to), so we always support it if we can.
We also set an additional flag CB_FILECLIP_NO_FILE_PATHS flag in the
capabilities because it seems to be necessary for the server to send the
"FileGroupDescriptorW" format to us. Otherwise the server will only send
the old CF_HDROP format which can't be handled well without enabled disk
drive redirection and a properly negotiated server-side temporary
directory.
xf_cliprdr_server_file_contents_request() handles the
CLIPRDR_FILECONTENTS_REQUEST which is sent by the server to retrieve the
size or data of a single file. The server can only request one of these
things at a time, do confirm this.
The actual handling of the requests is done by wClipboardDelegate. In
order to handle its resposes we register a bunch of our own callbacks
which simply pass the responses to the server.
In case of an error we should always send a CB_RESPONSE_FAIL to the
server. If we do not then the file transfer progress dialog may end up
hanging in the remote session forever until the user logs out or kills
explorer.exe. We do wnat this.
To handle a new format we should first be able to transform the format
name from the local clipboard owner into its remote representation. In
our case this will be trasforming the "text/uri-list" target into the
"FileGroupDescriptorW" named format.
Add CB_FORMAT_TEXTURILIST to identify the local format by its ID during
the data conversion step. This numeric ID has nothing to do with the ID
which will be sent to server. It's a bit weird, but that's how XFreeRDP
works.
After that add a new client format with this ID and appropriate local
and remote format names (in atom and formatName fields respectively).
Do this only if wClipboard actually supports "text/uri-list" format.
(It could fail to initialize the local file subsystem, in which case
it will fail all file-related requests and there would be no point in
advertising the file format support in the first place.)
Finally, handle the actual format data request for a new named format
in xf_cliprdr_process_requested_data(). Remember to convert the
FILEDESCRIPTOR array we receive from wClipboard into the
CLIPRDR_FILELIST expected by the server. Also take care to not leak
memory during this conversion.
Note that this handles only the CLIPRDR_FORMAT_DATA_REQUEST. The server
is still not able to retrieve the file content as this is done via a
separate request-reply sequence.
Unify error handling in xf_clipboard_new() by using a common cleanup
code path. This fixes a leak of clipboard->system and format names when
an error occurs during initialization.
Also reformat the code to fit into 100 column limit without this line
break insanity and with improved readability.
I do not particularly like to use a variable with such a descriptive
name 'n' far away in the error handling part, but its short name is
kinda important for readability so let's keep it as is.
One important point in the cliprdr protocol is that the peers are not
allowed to request file sizes and ranges if the clipboard content
changes. File locking should be used to gain this ability. However, our
file list is still accessible after new data is set into wClipboard.
Catch this error by storing the sequence number of the file list when it
is set and checking that it is still in effect at the time when the
client requests file sizes or ranges. There is a small chance of false
positives when the sequence number overflows, but I guess we can safely
ignore it.
This is another bunch of callbacks which provide the file contents to
the clients. We jump through some extra hoops in order to have more
pleasant user experience.
Simple stuff goes first. The file offset (or position) is split into the
low and high parts because this is the format in which the clients
receive the request from the server. They can simply copy the values as
is into the struct without repackaging them (which we do instead in the
end to get a 64-bit off_t).
Another thing is that we try to minimize the number of lseek() calls and
to keep as few file descriptors open as possible. We cannot open all the
files at once as there could be thousands of them and we'll run out of
the allowed number of the fds. However, the server can (in theory)
request the file ranges randomly so we need to be prepared for that. One
way to do that would be to always open the file before reading and close
it immediately afterwards. A dead simple solution with an acceptable
performance, but... some file systems do not support seeking, such as
FTP directories mounted over FUSE. However, they handle sequential
reading just fine *and* the server requests the data sequentially most
of the time so we can exploit this.
Thus open the file only once, during the first range request and keep
it open until the server reads all the data. In order to know how much
data is left we keep an accurate account of all reads and maintain the
file offset ourselves. This also allows us to avoid calling lseek() when
the file offset will not be effectively changed. However, if the server
requests some weird offset then we have no choice and will attempt
seeking. Unfortunately, we cannot tell whether it is a genuine failure
or the file system just does not support seeking, so we do not handle
the error further. (One workaround would be to reopen the file and keep
reading it until we're at the correct offset.) In this way we can
support sequential-only file systems if the server requests the contents
sequentially (and it does).
Also note that we do an fstat() right after opening the file in order to
have an accurate value of file size, for this exact file descriptor we
will be using. We should have it filled it by now, but just in case...
There is one more thing to explain. The cbRequested field specifies the
maximum number of bytes the server can handle, not the required number.
Usually this is some power-of-two number like 64 KB, based on the size
of the files on the clipboard. This is why posix_file_read_perform()
does not attempt to fill the whole buffer by repeatedly calling read()
if it has read less data than requested. The server can handle underruns
just fine (and this spares us from special-casing the EOF condition).
This is an example of wClipboardDelegate method implementation. POSIX
subsystem uses synchronous methods, but the interface can be used for
asynchronous request processing as well. The client should call a
Client* callback to request some action and the wClipboard will process
the request and report the result by calling an approriate Clipboard*
callback. Usually there will be two callbacks: one for reporting success
and one to report errors.
All callbacks have at least two arguments: the wClipboardDelegate itself
to pass the system context, and the wClipboard*Request structure with
the arguments to pass the call context. The request context is also
passed to the result callbacks by wClipboard so that the client can
match up the result with its previous request.
The fields of wClipboard*Request structures are heavily influenced by
the MS-RDPECLIP spec and mirror the respective fields of
CLIPRDR_FILECONTENTS_REQUEST. wClipboard should not depend on
MS-RDPECLIP, that's the reason we don't use CLIPRDR_FILECONTENTS_REQUEST
directly. However, I believe that we should not have void* fields in the
request structs so that they can be easily copied around if needed.
This is why have the weird 'streamId' field there which has nothing to
do with wClipboard and will be used only by the clients when sending
replies to the server.
Return values of the callbacks are to be used for reporting errors with
processing the request or reply per se, not for errors encountered while
performing the action requested. Thus, for example, we return NO_ERROR
from posix_file_request_size() even when we fail to report the result to
the client, because we have successfully performed the request and do
not care if the client could not handle our reply for some reason.
Also note that setup_delegate() fills in dummy implementations of
Clipboard* reply callbacks so that we do not crash in case the client
does not fill them and do not have to perform paranoid NULL checks
before calling every single callback.
This is the thing which will be used by clients to request file sizes
and ranges from wClipboard and by wClipboard to report the results of
the requests to the clients.
wClipboard and the client will fill in the (currently absent) callbacks
with their implementations of the request-report interface and will be
using them accordingly.
Initially I thought that wClipbardDelegate would be dynamically
allocated by the client and set into wClipboard (as this would be the
case with a delegate interface implementation in OOP langauges), but
after some thought I ended up with storing the delegate in wClipboard
and using the 'void* custom' field for client-private data.
So the idea is for the subsystem to fill in its callbacks during
wClipboard construction and for the client to get access to
wClipboardDelegate with a getter and fill in its callbacks during its
clipboard initialization. The subsystem will use wClipboard* pointer to
access its data and the client will have its void* pointer to store its
context.
text/uri-list contains only the files which were immediately selected by
the user. However, we need to enumerate *all* files and directories to
be pasted in CLIPRDR_FILELIST. Thus we need to walk through the
directories and add their content to the file list as well.
We use readdir() function to traverse the directory entries. It has more
sane interface than readdir_r(), but lacks (standardized) thread-safety
guarantees. However, most C liraries guarantee that so we can use it.
There is no compile-time check because it cannot be made robust. You
deserve a crash here if you are using a C library developed by people
who cannot keep their unhealthy addiction to global state under control.
Note that recursive traversal is also a good opportunity to maintain
good remote names. We just need to concatenate the directory paths and
file names correctly.
However, this recursion has one caveat: it is not bounded, so if the
file system contains a loop then we will crash due to a stack overflow.
We could track symlink loops (and hardlinks too if we try hard) to avoid
the crash, but I think it's not a common thing to do so we can ignore
this possibility.
Finally we can add a file to the file list once we have got its local
file name decoded. The interesting part here is what we use for the
remote name.
Suppose the user has selected two files in different directories. In
this case we end up receiving a text/uri-list like this:
file:///home/bob/foo/a
file:///home/bob/bar/b
We'd expect to see "a" and "b" pasted into the remote session, so that's
what we should use for the remote names: the base names of the files.
These are the parts from the end up to the last directory delimiter.
One tricky point here is that Windows expects the file names to be
encoded in Unicode, but POSIX does not specify any particular encoding
for file names. Operating systems and file systems generally handle the
file names as mostly opaque bytes strings and do not really care what
encoding is used there. There is no portable API to get the encoding,
it's entirely up to the users and the software they use to correctly
interpret the file names. But we need to do something here.
As of 2017, the most widely used encoding for file names is UTF-8. While
there are marginal communities which stick to codepages for legacy
reasons, we can safely assume that most of the time the file names will
be encoded in UTF-8. In fact, popular desktop environments like GNOME
also assume this. So that's what we will do here as well.
Nothing really interesting here, it's exactly what it says on the tin.
The percent-encoding is specified by RFC 3986. And we take care to
detect invalid encodings.
Now we start handling the actual format data. As the first step we need
to convert the text/uri-list data into the list of file names. Each file
or directory the user selects to copy is represented with a URI, and the
whole list looks like this:
file:///home/bob/text-file
file:///home/bob/a-directory
file:///home/bob/white%20space
The MIME format is actually specified by RFC 2483. As said in the
comments, we allow some slack for other applications: they can use
singular LF and CR as line terminators, and we also will handle missing
terminator for the last line (some applications actually do this, but I
can't recall which ones at the moment).
We will handle only the file:// URI scheme because these refer to local
filesystem paths. It is possible for text/uri-list to contain URIs with
other schemes when the user selects files from remote filesystems (like
an FTP server or an SMB share connect to from a file manager). We cannot
pass such paths to open() and for some reason the file managers use the
remote URIs even when the remote filesystems are actually mapped to the
local filesystem via FUSE. Therefore we restrict ourselves to handling
only file://.
Now we do the actual conversion of a list of struct posix_files into an
array of FILEDESCRIPTORs. In order to correctly fill in the fields we
need to know the size of the file and whether it is a directory. This
can be looked up by stat() call. Do this during struct posix_file
construction and cache the values for later use (we will need them).
Define _FILE_OFFSET_BITS to make off_t a 64-bit value and to call
appropriate functions when we write stat() in the code. FILEDESCRIPTOR
and cliprdr protocol expect the file sizes to be 64-bit so we can
provide accurate information with that. Take care to define it before
including any system headers ("config.h" contains only defines).
Also take care to not overrun the file name buffer. Windows has a hard
cap of 260 Unicode characters for the full file name, including the
terminating null character.
Here you can see an outline of our approach to handling file lists put
on the clipboard. Typical usage of wClipboard by the clients sums up to
doing a ClipboardSetData() call followed by a ClipboardGetData() call
with appropriate format ID passed to them. Thus for files we would
expect the clients to first set the local format (like "text/uri-list")
and then to get the remote format (the "FileGroupDescriptorW").
MS-RDPECLIP has a concept of locally-stored list of files on the
clipboard. This is modeled by clipboard->localFiles ArrayList. We need
to populate this list before we serialize it into CLIPRDR_FILELIST and
send it to the server. The easiest way to achieve this is a bit hacky,
but it works: we populate the file list from inside the synthesizer
callback registered for text/uri-list -> FileGroupDescriptorW
conversion.
So the client would first set the data it received from local clipboard
as "text/uri-list" format, then it gets a "FileGroupDescriptorW" format,
during that conversion we will prepare to serve file content requests,
and in the end we provide a FILEDESCRIPTOR array to the client as the
conversion result. The client will then serialize the array into
CLIPRDR_FILELIST and sent it to the server. (We cannot do serialization
in WinPR as WinPR should not know about cliprdr and its data formats.)
The subsystems are expected to store their private structures in the
clipboard->localFiles array. POSIX subsystem uses struct posix_file
which currently has bare minimum of fields: the local file name (for
open() and the like) and the remote file name (the one to put into
FILEDESCRIPTOR).
This adds some initial skeleton for local file subsystems of wClipboard.
The idea is to delegate handling of local file formats to dedicated
subsystems selected at runtime based on the compiled-in support code.
This is somewhat similar to the approach used by audin, rdpsnd, rdpgfx
channels in FreeRDP.
Only one subsystem is actually used by wClipboard during runtime. It is
selected by the ClipboardInitLocalFileSubsystem() function which will
try initializing the compiled-in subsystems in the preferred order. Thus
when adding new subsystems one must make sure to 1) return as soon as
any initialization succeeds, 2) leave wClipboard in usable state if the
initialization fails.
A POSIX file subsystem is added as a pioneer. It will handle local file
format "text/uri-list" and will use POSIX API to access the files. This
is the combination one would expect to be supported by Linux systems
which can run the XFreeRDP client.
The POSIX subsystem is enabled only when CMake detects <unistd.h> as
available. This is the core POSIX include file so we can reasonably
expect the rest of the POSIX API to be available along with that file.
We also define a new configuration option WITH_DEBUG_WCLIPBOARD which
will be used to guard some debug-only verbose logging in wClipboad.
Unify error handling in ClipboardInitFormats() and actually handle the
return value of ClipboardInitSynthesizers(). Currently it always returns
TRUE, but this may change, so we'd better be clean.
Declare 'formatName' in wClipboardFormat as non-const. It is customary
in C to declare owned pointers as non-const because various deallocation
functions like free() take non-const pointers as arguments. Furthermore,
const char* is tightly associated with "string literals" which must not
be freed. Thus declaring this field as non-const is more accurate, and
removes that ugly void* cast from ClipboardInitFormats().
Unify error handling in ClipboardCreate(). The cleanup snippet should
not be repeated as it's prone to errors, like leaking the allocation of
clipboard->formats when ClipboardInitFormats() fails. Unified error
handling makes it much harder to forget resource cleanup on errors.
The format is described in MS-RDPECLIP 2.2.5.2.3 Packed File List
(CLIPRDR_FILELIST). These functions handle conversion between the
on-the-wire data from cliprdr and arrays of FILEDESCRIPTOR structs.
FILETIME handling is a bit wacky, but that's what we currently have.
The flags are defined by MS-RDPECLIP 2.2.5.2.3.1 File Descriptor
(CLIPRDR_FILEDESCRIPTOR) as well as by 'File Attribute Constants'
in WinAPI reference [1].
The idea is to delegate FILEDESCRIPTOR format processing to WinPR
instead of cliprdr channel, so move the struct definition there. The
definition used by cliprdr protocol is identical but with some fields
treated as reserved.
The defintions are placed into <winpr/shell.h> as FileGroupDescriptorW
is a shell clipboard format.
Also remove the definition of CLIPRDR_FILELIST. The clients would be
using WinPR to handle the file clipping, so CLIPRDR_FILELIST does not
have to be handled explicitly. The clients will have serialization and
deserialization functions to handle CLIPRDR_FILELIST.
[1]: https://msdn.microsoft.com/en-us/library/windows/desktop/gg258117(v=vs.85).aspx
Since this comes via a Wire-To-Surface-2 PDU we don't have
any left/top/right/bottom destination values.
The current code has always dealt with zeros when updating the
invalid region which resulted in black rectangles.
The correct update region is determined during decompression.
