mirrors/postgres
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Discuss LOCALE differences as a reason for regression test failure.

Browse Source
This commit is contained in:
Tom Lane 2001-03-20 00:09:36 +00:00
parent 68bc848766
commit ca03b551fc
2 changed files with 71 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
doc/src/sgml/regress.sgml
View File

@ -1,4 +1,4 @@
<!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.16 2001/01/02 05:56:02 tgl Exp $ -->
<!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.17 2001/03/20 00:09:36 tgl Exp $ -->
<chapter id="regress">
<title id="regress-title">Regression Tests</title>
@ -49,7 +49,7 @@
<screen>
<computeroutput>
======================
All 75 tests passed.
All 76 tests passed.
======================
</computeroutput>
</screen>
@ -64,6 +64,7 @@
If you already did the build as root, you do not have to start all
over. Instead, make the regression test directory writable by
some other user, log in as that user, and restart the tests.
For example,
<screen>
<prompt>root# </prompt><userinput>chmod -R a+w src/test/regress</userinput>
<prompt>root# </prompt><userinput>su - joeuser</userinput>
@ -100,8 +101,9 @@
<screen>
<prompt>$ </prompt><userinput>gmake installcheck</userinput>
</screen>
The server is expected to be running on the local host with the
default port number.
The tests will expect to contact the server at the local host and the
default port number, unless directed otherwise by PGHOST and PGPORT
environment variables.
</para>
<sect1 id="regress-evaluation">
@ -111,8 +113,8 @@
Some properly installed and fully functional
<productname>PostgreSQL</productname> installations can
<quote>fail</quote> some of these regression tests due to
artifacts of floating point representation and time zone
support. The tests are currently evaluated using a simple
platform-specific artifacts such as varying floating point representation
and time zone support. The tests are currently evaluated using a simple
<application>diff</application> comparison against the outputs
generated on a reference system, so the results are sensitive to
small system differences. When a test is reported as
@ -149,6 +151,29 @@
</para>
</sect2>
<sect2>
<title>Locale differences</title>
<para>
The tests expect to run in plain <quote>C</quote> locale. This
should not cause any problems when you run the tests against a
temporary installation, since the regression test driver takes care
to start the server in C locale. However, if you run the tests
against an already-installed server that is using non-C locale settings,
you may see differences caused by varying rules for string sort order,
formatting of numeric and monetary values, and so forth.
</para>
<para>
In some locales the resulting differences are small and easily checked by
inspection. However, in a locale that changes the rules for formatting
of numeric values (typically by swapping the usage of commas and
decimal points), entry of some data values will fail, resulting in
extensive differences later in the tests where the missing data values
are supposed to be used.
</para>
</sect2>
<sect2>
<title>Date and time differences</title>
@ -262,13 +287,14 @@ according to the letter of the SQL spec. In practice, since we are
looking at the same queries being executed on the same data by the same
software, we usually get the same result ordering on all platforms, and
so the lack of ORDER BY isn't a problem. Some queries do exhibit
cross-platform ordering differences, however.
cross-platform ordering differences, however. (Ordering differences
can also be triggered by non-C locale settings.)
</para>
<para>
Therefore, if you see an ordering difference, it's not something to
worry about (unless the query does have an ORDER BY that your result
is violating). But please report it anyway, so that we can add an
worry about, unless the query does have an ORDER BY that your result
is violating. But please report it anyway, so that we can add an
ORDER BY to that particular query and thereby eliminate the bogus
<quote>failure</quote> in future releases.
</para>

52
src/test/regress/README
View File

@ -31,7 +31,7 @@ user-defined trigger functions, and then run the test driver
script. At the end you should see something like
======================
All 75 tests passed.
All 76 tests passed.
======================
or otherwise a note about what tests failed. See the section called
@ -42,7 +42,7 @@ Test Evaluation below for more.
root). If you already did the build as root, you do not have to
start all over. Instead, make the regression test directory
writable by some other user, log in as that user, and restart the
tests.
tests. For example,
root# chmod -R a+w src/test/regress
root# su - joeuser
@ -67,21 +67,23 @@ the server, then type
$ gmake installcheck
The server is expected to be running on the local host with the
default port number.
The tests will expect to contact the server at the local host and the
default port number, unless directed otherwise by PGHOST and PGPORT
environment variables.
Test Evaluation
Some properly installed and fully functional PostgreSQL installations
can "fail" some of these regression tests due to artifacts of floating
point representation and time zone support. The tests are currently
evaluated using a simple diff comparison against the outputs generated
on a reference system, so the results are sensitive to small system
differences. When a test is reported as "failed", always examine the
differences between expected and actual results; you may well find
that the differences are not significant. Nonetheless, we still strive
to maintain accurate reference files across all supported platforms,
so it can be expected that all tests pass.
can "fail" some of these regression tests due to platform-specific
artifacts such as varying floating point representation and time zone
support. The tests are currently evaluated using a simple diff
comparison against the outputs generated on a reference system, so the
results are sensitive to small system differences. When a test is
reported as "failed", always examine the differences between expected
and actual results; you may well find that the differences are not
significant. Nonetheless, we still strive to maintain accurate reference
files across all supported platforms, so it can be expected that all
tests pass.
The actual outputs of the regression tests are in files in the
src/test/regress/results directory. The test script uses diff to
@ -99,6 +101,23 @@ messages may vary between platforms, but should reflect similar
information. These differences in messages will result in a "failed"
regression test which can be validated by inspection.
Locale differences
The tests expect to run in plain "C" locale. This should not cause any
problems when you run the tests against a temporary installation, since
the regression test driver takes care to start the server in C locale.
However, if you run the tests against an already-installed server that
is using non-C locale settings, you may see differences caused by
varying rules for string sort order, formatting of numeric and monetary
values, and so forth.
In some locales the resulting differences are small and easily checked by
inspection. However, in a locale that changes the rules for formatting
of numeric values (typically by swapping the usage of commas and
decimal points), entry of some data values will fail, resulting in
extensive differences later in the tests where the missing data values
are supposed to be used.
Date and time differences
Most of the date and time results are dependent on the time zone
@ -177,11 +196,12 @@ according to the letter of the SQL spec. In practice, since we are
looking at the same queries being executed on the same data by the same
software, we usually get the same result ordering on all platforms, and
so the lack of ORDER BY isn't a problem. Some queries do exhibit
cross-platform ordering differences, however.
cross-platform ordering differences, however. (Ordering differences
can also be triggered by non-C locale settings.)
Therefore, if you see an ordering difference, it's not something to
worry about (unless the query does have an ORDER BY that your result
is violating). But please report it anyway, so that we can add an
worry about, unless the query does have an ORDER BY that your result
is violating. But please report it anyway, so that we can add an
ORDER BY to that particular query and thereby eliminate the bogus
"failure" in future releases.