mirrors/postgres
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improve code clarity in epilogue of UTF-8 verification fast path

Browse Source 
The previous coding was correct, but the style and commentary were a bit
vague about which operations had to happen, in what circumstances, and
in what order. Rearrange so that the epilogue does nothing in the DFA END
state. That allows turning some conditional statements in the backtracking
logic into asserts. With that, we can be more explicit about needing
to backtrack at least one byte in non-END states to ensure checking the
current byte sequence in the slow path. No change to the regression tests,
since they should be able catch deficiencies here already.

In passing, improve the comments around DFA states where the first
continuation byte has a restricted range.
This commit is contained in:
John Naylor 2022-01-17 22:53:50 -05:00
parent 9007d4ea77
commit d3f45323bb
1 changed files with 25 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
src/common/wchar.c
View File

@ -1807,12 +1807,11 @@ pg_utf8_verifychar(const unsigned char *s, int len)
#define CS1 16
#define CS2 1
#define CS3 5
/* Leading byte was E0/ED, expect 1 more continuation byte */
#define P3A 6
#define P3B 20
/* Leading byte was F0/F4, expect 2 more continuation bytes */
#define P4A 25
#define P4B 30
/* Partial states, where the first continuation byte has a restricted range */
#define P3A 6 /* Lead was E0, check for 3-byte overlong */
#define P3B 20 /* Lead was ED, check for surrogate */
#define P4A 25 /* Lead was F0, check for 4-byte overlong */
#define P4B 30 /* Lead was F4, check for too-large */
/* Begin and End are the same state */
#define END BGN
@ -1941,31 +1940,32 @@ pg_utf8_verifystr(const unsigned char *s, int len)
len -= STRIDE_LENGTH;
}
/*
* The error state persists, so we only need to check for it here. In
* case of error we start over from the beginning with the slow path
* so we can count the valid bytes.
*/
/* The error state persists, so we only need to check for it here. */
if (state == ERR)
{
/*
* Start over from the beginning with the slow path so we can
* count the valid bytes.
*/
len = orig_len;
s = start;
}
/*
* We treat all other states as success, but it's possible the fast
* path exited in the middle of a multibyte sequence, since that
* wouldn't have caused an error. Before checking the remaining bytes,
* walk backwards to find the last byte that could have been the start
* of a valid sequence.
*/
while (s > start)
else if (state != END)
{
s--;
len++;
if (!IS_HIGHBIT_SET(*s) || pg_utf_mblen(s) > 1)
break;
/*
* The fast path exited in the middle of a multibyte sequence.
* Walk backwards to find the leading byte so that the slow path
* can resume checking from there. We must always backtrack at
* least one byte, since the current byte could be e.g. an ASCII
* byte after a 2-byte lead, which is invalid.
*/
do
{
Assert(s > start);
s--;
len++;
Assert(IS_HIGHBIT_SET(*s));
} while (pg_utf_mblen(s) <= 1);
}
}