mirrors/postgres
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Discuss LOCALE differences as a reason for regression test failure.

Browse Source
This commit is contained in:
Tom Lane 2001-03-20 00:09:36 +00:00
parent 68bc848766
commit ca03b551fc
2 changed files with 71 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
doc/src/sgml/regress.sgml
View File

@ -1,4 +1,4 @@
<!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.16 2001/01/02 05:56:02 tgl Exp $ --> <!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.17 2001/03/20 00:09:36 tgl Exp $ -->
<chapter id="regress"> <chapter id="regress">
<title id="regress-title">Regression Tests</title> <title id="regress-title">Regression Tests</title>
@ -49,7 +49,7 @@
<screen> <screen>
<computeroutput> <computeroutput>
====================== ======================
All 75 tests passed. All 76 tests passed.
====================== ======================
</computeroutput> </computeroutput>
</screen> </screen>
@ -64,6 +64,7 @@
If you already did the build as root, you do not have to start all If you already did the build as root, you do not have to start all
over. Instead, make the regression test directory writable by over. Instead, make the regression test directory writable by
some other user, log in as that user, and restart the tests. some other user, log in as that user, and restart the tests.
For example,
<screen> <screen>
<prompt>root# </prompt><userinput>chmod -R a+w src/test/regress</userinput> <prompt>root# </prompt><userinput>chmod -R a+w src/test/regress</userinput>
<prompt>root# </prompt><userinput>su - joeuser</userinput> <prompt>root# </prompt><userinput>su - joeuser</userinput>
@ -100,8 +101,9 @@
<screen> <screen>
<prompt>$ </prompt><userinput>gmake installcheck</userinput> <prompt>$ </prompt><userinput>gmake installcheck</userinput>
</screen> </screen>
The server is expected to be running on the local host with the The tests will expect to contact the server at the local host and the
default port number. default port number, unless directed otherwise by PGHOST and PGPORT
environment variables.
</para> </para>
<sect1 id="regress-evaluation"> <sect1 id="regress-evaluation">
@ -111,8 +113,8 @@
Some properly installed and fully functional Some properly installed and fully functional
<productname>PostgreSQL</productname> installations can <productname>PostgreSQL</productname> installations can
<quote>fail</quote> some of these regression tests due to <quote>fail</quote> some of these regression tests due to
artifacts of floating point representation and time zone platform-specific artifacts such as varying floating point representation
support. The tests are currently evaluated using a simple and time zone support. The tests are currently evaluated using a simple
<application>diff</application> comparison against the outputs <application>diff</application> comparison against the outputs
generated on a reference system, so the results are sensitive to generated on a reference system, so the results are sensitive to
small system differences. When a test is reported as small system differences. When a test is reported as
@ -149,6 +151,29 @@
</para> </para>
</sect2> </sect2>
<sect2>
<title>Locale differences</title>
<para>
The tests expect to run in plain <quote>C</quote> locale. This
should not cause any problems when you run the tests against a
temporary installation, since the regression test driver takes care
to start the server in C locale. However, if you run the tests
against an already-installed server that is using non-C locale settings,
you may see differences caused by varying rules for string sort order,
formatting of numeric and monetary values, and so forth.
</para>
<para>
In some locales the resulting differences are small and easily checked by
inspection. However, in a locale that changes the rules for formatting
of numeric values (typically by swapping the usage of commas and
decimal points), entry of some data values will fail, resulting in
extensive differences later in the tests where the missing data values
are supposed to be used.
</para>
</sect2>
<sect2> <sect2>
<title>Date and time differences</title> <title>Date and time differences</title>
@ -262,13 +287,14 @@ according to the letter of the SQL spec. In practice, since we are
looking at the same queries being executed on the same data by the same looking at the same queries being executed on the same data by the same
software, we usually get the same result ordering on all platforms, and software, we usually get the same result ordering on all platforms, and
so the lack of ORDER BY isn't a problem. Some queries do exhibit so the lack of ORDER BY isn't a problem. Some queries do exhibit
cross-platform ordering differences, however. cross-platform ordering differences, however. (Ordering differences
can also be triggered by non-C locale settings.)
</para> </para>
<para> <para>
Therefore, if you see an ordering difference, it's not something to Therefore, if you see an ordering difference, it's not something to
worry about (unless the query does have an ORDER BY that your result worry about, unless the query does have an ORDER BY that your result
is violating). But please report it anyway, so that we can add an is violating. But please report it anyway, so that we can add an
ORDER BY to that particular query and thereby eliminate the bogus ORDER BY to that particular query and thereby eliminate the bogus
<quote>failure</quote> in future releases. <quote>failure</quote> in future releases.
</para> </para>

52
src/test/regress/README
View File

@ -31,7 +31,7 @@ user-defined trigger functions, and then run the test driver
script. At the end you should see something like script. At the end you should see something like
====================== ======================
All 75 tests passed. All 76 tests passed.
====================== ======================
or otherwise a note about what tests failed. See the section called or otherwise a note about what tests failed. See the section called
@ -42,7 +42,7 @@ Test Evaluation below for more.
root). If you already did the build as root, you do not have to root). If you already did the build as root, you do not have to
start all over. Instead, make the regression test directory start all over. Instead, make the regression test directory
writable by some other user, log in as that user, and restart the writable by some other user, log in as that user, and restart the
tests. tests. For example,
root# chmod -R a+w src/test/regress root# chmod -R a+w src/test/regress
root# su - joeuser root# su - joeuser
@ -67,21 +67,23 @@ the server, then type
$ gmake installcheck $ gmake installcheck
The server is expected to be running on the local host with the The tests will expect to contact the server at the local host and the
default port number. default port number, unless directed otherwise by PGHOST and PGPORT
environment variables.
Test Evaluation Test Evaluation
Some properly installed and fully functional PostgreSQL installations Some properly installed and fully functional PostgreSQL installations
can "fail" some of these regression tests due to artifacts of floating can "fail" some of these regression tests due to platform-specific
point representation and time zone support. The tests are currently artifacts such as varying floating point representation and time zone
evaluated using a simple diff comparison against the outputs generated support. The tests are currently evaluated using a simple diff
on a reference system, so the results are sensitive to small system comparison against the outputs generated on a reference system, so the
differences. When a test is reported as "failed", always examine the results are sensitive to small system differences. When a test is
differences between expected and actual results; you may well find reported as "failed", always examine the differences between expected
that the differences are not significant. Nonetheless, we still strive and actual results; you may well find that the differences are not
to maintain accurate reference files across all supported platforms, significant. Nonetheless, we still strive to maintain accurate reference
so it can be expected that all tests pass. files across all supported platforms, so it can be expected that all
tests pass.
The actual outputs of the regression tests are in files in the The actual outputs of the regression tests are in files in the
src/test/regress/results directory. The test script uses diff to src/test/regress/results directory. The test script uses diff to
@ -99,6 +101,23 @@ messages may vary between platforms, but should reflect similar
information. These differences in messages will result in a "failed" information. These differences in messages will result in a "failed"
regression test which can be validated by inspection. regression test which can be validated by inspection.
Locale differences
The tests expect to run in plain "C" locale. This should not cause any
problems when you run the tests against a temporary installation, since
the regression test driver takes care to start the server in C locale.
However, if you run the tests against an already-installed server that
is using non-C locale settings, you may see differences caused by
varying rules for string sort order, formatting of numeric and monetary
values, and so forth.
In some locales the resulting differences are small and easily checked by
inspection. However, in a locale that changes the rules for formatting
of numeric values (typically by swapping the usage of commas and
decimal points), entry of some data values will fail, resulting in
extensive differences later in the tests where the missing data values
are supposed to be used.
Date and time differences Date and time differences
Most of the date and time results are dependent on the time zone Most of the date and time results are dependent on the time zone
@ -177,11 +196,12 @@ according to the letter of the SQL spec. In practice, since we are
looking at the same queries being executed on the same data by the same looking at the same queries being executed on the same data by the same
software, we usually get the same result ordering on all platforms, and software, we usually get the same result ordering on all platforms, and
so the lack of ORDER BY isn't a problem. Some queries do exhibit so the lack of ORDER BY isn't a problem. Some queries do exhibit
cross-platform ordering differences, however. cross-platform ordering differences, however. (Ordering differences
can also be triggered by non-C locale settings.)
Therefore, if you see an ordering difference, it's not something to Therefore, if you see an ordering difference, it's not something to
worry about (unless the query does have an ORDER BY that your result worry about, unless the query does have an ORDER BY that your result
is violating). But please report it anyway, so that we can add an is violating. But please report it anyway, so that we can add an
ORDER BY to that particular query and thereby eliminate the bogus ORDER BY to that particular query and thereby eliminate the bogus
"failure" in future releases. "failure" in future releases.