Commit Graph

55873 Commits

Author SHA1 Message Date
Tom Lane e3ac85014e Support PlaceHolderVars in MERGE actions.
preprocess_targetlist thought PHVs couldn't appear here.
It was mistaken, as per report from Önder Kalacı.

Surveying other pull_var_clause calls, I noted no similar errors,
but I did notice that qual_is_pushdown_safe's assertion about
!contain_window_function was pointless, because the following
pull_var_clause call would complain about them anyway.  In HEAD
only, remove the redundant Assert and improve the commentary.

Discussion: https://postgr.es/m/CACawEhUuum-gC_2S3sXLTcsk7bUSPSHOD+g1ZpfKaDK-KKPPWA@mail.gmail.com
2023-03-15 11:59:18 -04:00
Alvaro Herrera a0137388cb
doc: Add lists of modules trusted/obsolete
Author: Karl Pinc <kop@karlpinc.com>
Discussion: https://postgr.es/m/20230102180015.372995a9@slate.karlpinc.com
2023-03-15 09:32:44 +01:00
Thomas Munro a948e49e2e Use nanosleep() to implement pg_usleep().
The previous coding based on select() had commentary about historical
portability concerns.  Use POSIX nanosleep() instead.

This has independently been suggested a couple of times before, but
never managed to stick.  Since recent and proposed work removes other
uses of select(), and associated code and comments relating to its
non-portable interaction with signals, it seems like a good time to tidy
up this case, too.

Also modernize the explanation of why WaitLatch() is a better way to
wait.

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Suggested-by: Paul Guo <paulguo@gmail.com>
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAAKRu_b-q0hXCBUCAATh0Z4Zi6UkiC0k2DFgoD3nC-r3SkR3tg%40mail.gmail.com
Discussion: https://postgr.es/m/CABQrizfxpBLZT5mZeE0js5oCh1tqEWvcGF3vMRCv5P-RwUY5dQ@mail.gmail.com
Discussion: https://postgr.es/m/4902.1552349020@sss.pgh.pa.us
2023-03-15 17:57:12 +13:00
Thomas Munro e4da2a44c1 Update obsolete comment about pg_usleep() accuracy.
There are still some systems that use traditional tick-based sleep
timing, but many including Linux, FreeBSD and macOS started using high
resolution timer hardware more directly a decade or two ago.  Update our
comment about that.  Also highlight that Windows is like the older
Unixen in that respect.

Author: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2BogAon8_V223Ldv6taPR2uKH3X_UJ_A7LJAf3-VRARPA%40mail.gmail.com
2023-03-15 17:23:02 +13:00
Amit Kapila 805b821e77 Add the testcases for 89e46da5e5.
Forgot to add new testcases in commit 89e46da5e5.

Author: Onder Kalaci, Amit Kapila
Reviewed-by: Peter Smith, Shi yu, Hou Zhijie, Vignesh C, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/CACawEhVLqmAAyPXdHEPv1ssU2c=dqOniiGz7G73HfyS7+nGV4w@mail.gmail.com
2023-03-15 09:31:44 +05:30
Michael Paquier 765f5df726 Improve WIN32 port of fstat() to detect more file types
The current implementation of _pgfstat64() is ineffective in detecting a
terminal handle or an anonymous named pipe.  This commit improves our
port of fstat() to detect more efficiently such cases by relying on
GetFileType(), and returning more correct data when the type found is
either a FILE_TYPE_PIPE (_S_IFIFO) or a FILE_TYPE_CHAR (_S_IFCHR).

This is part of a more global fix to address failures when feeding the
output generated by pg_dump to pg_restore through a pipe, for example,
but not all of it.   We are also going to need to do something about
fseek() and ftello() which are not reliable on WIN32 for the same cases
where fstat() was incorrect.  Fixing fstat() is independent of the rest,
though, which is why both fixes are handled separately, and this is the
first part of it.

Reported-by: Daniel Watzinger
Author: Daniel Watzinger, Juan José Santamaría Flecha
Discussion: https://postgr.es/m/b1448cd7-871e-20e3-8398-895e2d1d3bf9@gmail.com
Backpatch-through: 14
2023-03-15 12:55:51 +09:00
Amit Kapila 89e46da5e5 Allow the use of indexes other than PK and REPLICA IDENTITY on the subscriber.
Using REPLICA IDENTITY FULL on the publisher can lead to a full table scan
per tuple change on the subscription when REPLICA IDENTITY or PK index is
not available. This makes REPLICA IDENTITY FULL impractical to use apart
from some small number of use cases.

This patch allows using indexes other than PRIMARY KEY or REPLICA
IDENTITY on the subscriber during apply of update/delete. The index that
can be used must be a btree index, not a partial index, and it must have
at least one column reference (i.e. cannot consist of only expressions).
We can uplift these restrictions in the future. There is no smart
mechanism to pick the index. If there is more than one index that
satisfies these requirements, we just pick the first one. We discussed
using some of the optimizer's low-level APIs for this but ruled it out
as that can be a maintenance burden in the long run.

This patch improves the performance in the vast majority of cases and the
improvement is proportional to the amount of data in the table. However,
there could be some regression in a small number of cases where the indexes
have a lot of duplicate and dead rows. It was discussed that those are
mostly impractical cases but we can provide a table or subscription level
option to disable this feature if required.

Author: Onder Kalaci, Amit Kapila
Reviewed-by: Peter Smith, Shi yu, Hou Zhijie, Vignesh C, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/CACawEhVLqmAAyPXdHEPv1ssU2c=dqOniiGz7G73HfyS7+nGV4w@mail.gmail.com
2023-03-15 08:49:04 +05:30
Thomas Munro 720de00af4 Fix fractional vacuum_cost_delay.
Commit 4753ef37 changed vacuum_delay_point() to use the WaitLatch() API,
to fix the problem that vacuum could keep running for a very long time
after the postmaster died.

Unfortunately, that broke commit caf626b2's support for fractional
vacuum_cost_delay, which shipped in PostgreSQL 12.  WaitLatch() works in
whole milliseconds.

For now, revert the change from commit 4753ef37, but add an explicit
check for postmaster death.  That's an extra system call on systems
other than Linux and FreeBSD, but that overhead doesn't matter much
considering that we willingly went to sleep and woke up again.  (In
later work, we might add higher resolution timeouts to the latch API so
that we could do this with our standard programming pattern, but that
wouldn't be back-patched.)

Back-patch to 14, where commit 4753ef37 arrived.

Reported-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b-q0hXCBUCAATh0Z4Zi6UkiC0k2DFgoD3nC-r3SkR3tg%40mail.gmail.com
2023-03-15 13:58:18 +13:00
Thomas Munro d41a178b3a Fix waitpid() emulation on Windows.
Our waitpid() emulation didn't prevent a PID from being recycled by the
OS before the call to waitpid().  The postmaster could finish up
tracking more than one child process with the same PID, and confuse
them.

Fix, by moving the guts of pgwin32_deadchild_callback() into waitpid(),
so that resources are released synchronously.  The process and PID
continue to exist until we close the process handle, which only happens
once we're ready to adjust our book-keeping of running children.

This seems to explain a couple of failures on CI.  It had never been
reported before, despite the code being as old as the Windows port.
Perhaps Windows started recycling PIDs more rapidly, or perhaps timing
changes due to commit 7389aad6 made it more likely to break.

Thanks to Alexander Lakhin for analysis and Andres Freund for tracking
down the root cause.

Back-patch to all supported branches.

Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20230208012852.bvkn2am4h4iqjogq%40awork3.anarazel.de
2023-03-15 13:24:47 +13:00
Tom Lane b081fe4199 Fix corner case bug in numeric to_char() some more.
The band-aid applied in commit f0bedf3e4 turns out to still need
some work: it made sure we didn't set Np->last_relevant too small
(to the left of the decimal point), but it didn't prevent setting
it too large (off the end of the partially-converted string).
This could result in fetching data beyond the end of the allocated
space, which with very bad luck could cause a SIGSEGV, though
I don't see any hazard of interesting memory disclosure.

Per bug #17839 from Thiago Nunes.  The bug's pretty ancient,
so back-patch to all supported versions.

Discussion: https://postgr.es/m/17839-aada50db24d7b0da@postgresql.org
2023-03-14 19:17:31 -04:00
Tom Lane a563c24c95 Allow pg_dump to include/exclude child tables automatically.
This patch adds new pg_dump switches
    --table-and-children=pattern
    --exclude-table-and-children=pattern
    --exclude-table-data-and-children=pattern
which act the same as the existing --table, --exclude-table, and
--exclude-table-data switches, except that any partitions or
inheritance child tables of the table(s) matching the pattern
are also included or excluded.

Gilles Darold, reviewed by Stéphane Tachoires

Discussion: https://postgr.es/m/5aa393b5-5f67-8447-b83e-544516990ee2@migops.com
2023-03-14 16:09:03 -04:00
Tom Lane 684ffac8c7 Remove unnecessary code in dependency_is_compatible_expression().
Scanning the expression for compatible Vars isn't really necessary,
because the subsequent match against StatisticExtInfo entries will
eliminate expressions containing other Vars just fine.  Moreover,
this code hadn't stopped to think about what to do with
PlaceHolderVars or Aggrefs in the clause; and at least for the PHV
case, that demonstrably leads to failures.  Rather than work out
whether it's reasonable to ignore those, let's just remove the
whole stanza.

Per report from Richard Guo.  Back-patch to v14 where this code
was added.

Discussion: https://postgr.es/m/CAMbWs48Mmvm-acGevXuwpB=g5JMqVSL6i9z5UaJyLGJqa-XPAA@mail.gmail.com
2023-03-14 11:10:45 -04:00
Daniel Gustafsson bdc8d803e8 doc: spell out full productname
Use PostgreSQL consistently for referring to the productname rather
than Postgres.  This also adds <productname> markup.

Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: "Jonathan S. Katz" <jkatz@postgresql.org>
Discussion: https://postgr.es/m/9C019644-9EA4-4B79-A52C-5C47A5B6B2DF@yesql.se
2023-03-14 14:41:55 +01:00
Michael Paquier 5c1b662807 Rework design of functions in pg_walinspect
This commit reworks a bit the set-returning functions of pg_walinspect,
making them more flexible regarding their end LSN:
- pg_get_wal_records_info()
- pg_get_wal_stats()
- pg_get_wal_block_info()

The end LSNs given to these functions is now handled so as a value
higher than the current LSN of the cluster (insert LSN for a primary, or
replay LSN for a standby) does not raise an error, giving more
flexibility to monitoring queries.  Instead, the functions return
results up to the current LSN, as found at the beginning of each
function call.

As an effect of that, pg_get_wal_records_info_till_end_of_wal() and
pg_get_wal_stats_till_end_of_wal() are now removed from 1.1, as the
existing, equivalent functions are able to offer the same
possibilities.

Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACU0_q-o4DSweyaW9NO1KBx-QkN6G_OzYQvpjf3CZVASkg@mail.gmail.com
2023-03-14 20:13:02 +09:00
Dean Rasheed d5d574146d Add support for the error functions erf() and erfc().
Expose the standard error functions as SQL-callable functions. These
are expected to be useful to people working with normal distributions,
and we use them here to test the distribution from random_normal().

Since these functions are defined in the POSIX and C99 standards, they
should in theory be available on all supported platforms. If that
turns out not to be the case, more work will be needed.

On all platforms tested so far, using extra_float_digits = -1 in the
regression tests is sufficient to allow for variations between
implementations. However, past experience has shown that there are
almost certainly going to be additional unexpected portability issues,
so these tests may well need further adjustments, based on the
buildfarm results.

Dean Rasheed, reviewed by Nathan Bossart and Thomas Munro.

Discussion: https://postgr.es/m/CAEZATCXv5fi7+Vu-POiyai+ucF95+YMcCMafxV+eZuN1B-=MkQ@mail.gmail.com
2023-03-14 09:17:36 +00:00
Michael Paquier 3a465cc678 libpq: Add support for require_auth to control authorized auth methods
The new connection parameter require_auth allows a libpq client to
define a list of comma-separated acceptable authentication types for use
with the server.  There is no negotiation: if the server does not
present one of the allowed authentication requests, the connection
attempt done by the client fails.

The following keywords can be defined in the list:
- password, for AUTH_REQ_PASSWORD.
- md5, for AUTH_REQ_MD5.
- gss, for AUTH_REQ_GSS[_CONT].
- sspi, for AUTH_REQ_SSPI and AUTH_REQ_GSS_CONT.
- scram-sha-256, for AUTH_REQ_SASL[_CONT|_FIN].
- creds, for AUTH_REQ_SCM_CREDS (perhaps this should be removed entirely
now).
- none, to control unauthenticated connections.

All the methods that can be defined in the list can be negated, like
"!password", in which case the server must NOT use the listed
authentication type.  The special method "none" allows/disallows the use
of unauthenticated connections (but it does not govern transport-level
authentication via TLS or GSSAPI).

Internally, the patch logic is tied to check_expected_areq(), that was
used for channel_binding, ensuring that an incoming request is
compatible with conn->require_auth.  It also introduces a new flag,
conn->client_finished_auth, which is set by various authentication
routines when the client side of the handshake is finished.  This
signals to check_expected_areq() that an AUTH_REQ_OK from the server is
expected, and allows the client to complain if the server bypasses
authentication entirely, with for example the reception of a too-early
AUTH_REQ_OK message.

Regression tests are added in authentication TAP tests for all the
keywords supported (except "creds", because it is around only for
compatibility reasons).  A new TAP script has been added for SSPI, as
there was no script dedicated to it yet.  It relies on SSPI being the
default authentication method on Windows, as set by pg_regress.

Author: Jacob Champion
Reviewed-by: Peter Eisentraut, David G. Johnston, Michael Paquier
Discussion: https://postgr.es/m/9e5a8ccddb8355ea9fa4b75a1e3a9edc88a70cd3.camel@vmware.com
2023-03-14 14:00:05 +09:00
Andres Freund 727400994d meson: fix openssl detection issues in 6a30027
When not detecting openssl via pkg-config, we'd error out if the headers
weren't found, even if -Dssl=auto. When detecting via pkg-config, but the
headers could not be found, we'd error out because the ssl_int variable would
not exist.

Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/20230313180432.GA246741@nathanxps13
2023-03-13 14:44:43 -07:00
Tom Lane 25a7812cd0 Fix JSON error reporting for many cases of erroneous string values.
The majority of error exit cases in json_lex_string() failed to
set lex->token_terminator, causing problems for the error context
reporting code: it would see token_terminator less than token_start
and do something more or less nuts.  In v14 and up the end result
could be as bad as a crash in report_json_context().  Older
versions accidentally avoided that fate; but all versions produce
error context lines that are far less useful than intended,
because they'd stop at the end of the prior token instead of
continuing to where the actually-bad input is.

To fix, invent some macros that make it less notationally painful
to do the right thing.  Also add documentation about what the
function is actually required to do; and in >= v14, add an assertion
in report_json_context about token_terminator being sufficiently
far advanced.

Per report from Nikolay Shaplov.  Back-patch to all supported
versions.

Discussion: https://postgr.es/m/7332649.x5DLKWyVIX@thinkpad-pgpro
2023-03-13 15:19:00 -04:00
Tom Lane 30dbdbe753 Fix failure to detect some cases of improperly-nested aggregates.
check_agg_arguments_walker() supposed that it needn't descend into
the arguments of a lower-level aggregate function, but this is
just wrong in the presence of multiple levels of sub-select.  The
oversight would lead to executor failures on queries that should
be rejected.  (Prior to v11, they actually were rejected, thanks
to a "redundant" execution-time check.)

Per bug #17835 from Anban Company.  Back-patch to all supported
branches.

Discussion: https://postgr.es/m/17835-4f29f3098b2d0ba4@postgresql.org
2023-03-13 12:40:28 -04:00
Andrew Dunstan 9f8377f7a2 Add a DEFAULT option to COPY FROM
This allows for a string which if an input field matches causes the
column's default value to be inserted. The advantage of this is that
the default can be inserted in some rows and not others, for which
non-default data is available.

The file_fdw extension is also modified to take allow use of this
option.

Israel Barth Rubio

Discussion: https://postgr.es/m/CAO_rXXAcqesk6DsvioOZ5zmeEmpUN5ktZf-9=9yu+DTr0Xr8Uw@mail.gmail.com
2023-03-13 10:01:56 -04:00
Dean Rasheed 7b14e20b12 Fix MERGE command tag for actions blocked by BEFORE ROW triggers.
This ensures that the row count in the command tag for a MERGE is
correctly computed in the case where UPDATEs or DELETEs are skipped
due to a BEFORE ROW trigger returning NULL (the INSERT case was
already handled correctly by ExecMergeNotMatched() calling
ExecInsert()).

Back-patch to v15, where MERGE was introduced.

Discussion: https://postgr.es/m/CAEZATCU8XEmR0JWKDtyb7iZ%3DqCffxS9uyJt0iOZ4TV4RT%2Bow1w%40mail.gmail.com
2023-03-13 11:12:20 +00:00
Dean Rasheed 9321c79c86 Fix concurrent update issues with MERGE.
If MERGE attempts an UPDATE or DELETE on a table with BEFORE ROW
triggers, or a cross-partition UPDATE (with or without triggers), and
a concurrent UPDATE or DELETE happens, the merge code would fail.

In some cases this would lead to a crash, while in others it would
cause the wrong merge action to be executed, or no action at all. The
immediate cause of the crash was the trigger code calling
ExecGetUpdateNewTuple() as part of the EPQ mechanism, which fails
because during a merge ri_projectNew is NULL, since merge has its own
per-action projection information, which ExecGetUpdateNewTuple() knows
nothing about.

Fix by arranging for the trigger code to exit early, returning the
TM_Result and TM_FailureData information, if a concurrent modification
is detected, allowing the merge code to do the necessary EPQ handling
in its own way. Similarly, prevent the cross-partition update code
from doing any EPQ processing for a merge, allowing the merge code to
work out what it needs to do.

This leads to a number of simplifications in nodeModifyTable.c. Most
notably, the ModifyTableContext->GetUpdateNewTuple() callback is no
longer needed, and mergeGetUpdateNewTuple() can be deleted, since
there is no longer any requirement for get-update-new-tuple during a
merge. Similarly, ModifyTableContext->cpUpdateRetrySlot is no longer
needed. Thus ExecGetUpdateNewTuple() and the retry_slot handling of
ExecCrossPartitionUpdate() can be restored to how they were in v14,
before the merge code was added, and ExecMergeMatched() no longer
needs any special-case handling for cross-partition updates.

While at it, tidy up ExecUpdateEpilogue() a bit, making it handle
recheckIndexes locally, rather than passing it in as a parameter,
ensuring that it is freed properly. This dates back to when it was
split off from ExecUpdate() to support merge.

Per bug #17809 from Alexander Lakhin, and follow-up investigation of
bug #17792, also from Alexander Lakhin.

Back-patch to v15, where MERGE was introduced, taking care to preserve
backwards-compatibility of the trigger API in v15 for any extensions
that might use it.

Discussion:
  https://postgr.es/m/17809-9e6650bef133f0fe%40postgresql.org
  https://postgr.es/m/17792-0f89452029662c36%40postgresql.org
2023-03-13 10:22:22 +00:00
Peter Eisentraut b2bd9a6796 Fix expected test output
For builds without lz4, for 208bf364a9.
2023-03-13 11:15:21 +01:00
Peter Eisentraut 208bf364a9 Remove incidental md5() function uses from main regression tests
Most of these calls were to generate some random data.  These can be
replaced by appropriately adapted sha256() calls.  To keep the diff
smaller, we wrap this into a helper function that produces the same
output format and length as the md5() call.

This will eventually allow these tests to pass in OpenSSL FIPS mode
(which does not allow MD5 use).

Similar work for other test suites will follow later.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/dbbd927f-ef1f-c9a1-4ec6-c759778ac852@enterprisedb.com
2023-03-13 10:53:28 +01:00
Peter Eisentraut d72900bded Improve support for UNICODE collation on older ICU
The recently added standard collation UNICODE (0d21d4b9bc) doesn't
give consistent results on some build farm members with old ICU
versions.  Apparently, the ICU locale specification 'und' (language
tag style) misbehaves on some older ICU versions.  Replacing it with
'' (ICU locale ID style) fixes it at least on some OS versions.  Let's
see what the build farm says.
2023-03-13 09:08:58 +01:00
Michael Paquier e0a09d4e35 Fix inconsistent error handling for GSS encryption in PQconnectPoll()
The error cases for TLS and GSS encryption were inconsistent.  After TLS
fails, the connection is marked as dead and follow-up calls of
PQconnectPoll() would return immediately, but GSS encryption was not
doing that, so the connection would still have been allowed to enter the
GSS handling code.  This was handled incorrectly when gssencmode was set
to "require".  "prefer" was working correctly, and this could not happen
under "disable" as GSS encryption would not be attempted.

This commit makes the error handling of GSS encryption on par with TLS
portion, fixing the case of gssencmode=require.

Reported-by: Jacob Champion
Author: Michael Paquier
Reviewed-by: Jacob Champion, Stephen Frost
Discussion: https://postgr.es/m/23787477-5fe1-a161-6d2a-e459f74c4713@timescale.com
Backpatch-through: 12
2023-03-13 16:36:20 +09:00
Peter Eisentraut bf32ec2256 cirrus/ccache: Use G rather than GB suffix
The former being the documented spelling.

Author: Justin Pryzby <pryzbyj@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/20230203142656.GA1653%40telsasoft.com
2023-03-13 07:23:45 +01:00
Peter Eisentraut 6a3002715e meson: Make auto the default of the ssl option
The 'ssl' option is of type 'combo', but we add a choice 'auto' that
simulates the behavior of a feature option.  This way, openssl is used
automatically by default if present, but we retain the ability to
potentially select another ssl library.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/ad65ffd1-a9a7-fda1-59c6-f7dc763c3051%40enterprisedb.com
2023-03-13 07:04:11 +01:00
Michael Paquier 1f282c24e4 Refactor and improve tests of pg_walinspect
The regression tests of pg_walinspect are reworked on a few aspects:
- Reorganization on the validation checks done for the start and end
LSNs on the six SQL functions currently available in 1.1.
- Addition of a few patterns doing bound checks for invalid start LSN,
invalid end LSN, and failures in reading LSN positions, for anything
that's been missing.
- Use of a consistent style across the whole, limiting blank lines
across the queries.
- Addition of a new test script for upgrades.  For the time being, this
is straight-forward with a check that the upgrade from 1.0 works
correctly.  This will be made more complicated once the interface of
this extension is reworked in 1.1 with a follow-up patch.

Most of the contents of this commit are extracted from a larger patch by
the same author, largely reorganized by me to minimize the differences
with the upcoming work aimed to lift the bound checks on the input LSNs
used by the functions of this extension.

Author: Bharath Rupireddy, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACU0_q-o4DSweyaW9NO1KBx-QkN6G_OzYQvpjf3CZVASkg@mail.gmail.com
2023-03-13 13:03:29 +09:00
Tom Lane 767c598954 Work around implementation restriction in adjust_appendrel_attrs.
adjust_appendrel_attrs can't transfer nullingrel labeling to a non-Var
translation expression (mainly because it's too late to wrap such an
expression in a PlaceHolderVar).  I'd supposed in commit 2489d76c4
that that restriction was unreachable because we'd not attempt to push
problematic clauses down to an appendrel child relation.  I forgot that
set_append_rel_size blindly converts all the parent rel's joininfo
clauses to child clauses, and that list could well contain clauses
from above a nulling outer join.

We might eventually have to devise a direct fix for this implementation
restriction, but for now it seems enough to filter out troublesome
clauses while constructing the child's joininfo list.  Such clauses
are certainly not useful while constructing paths for the child rel;
they'll have to be applied later when we join the completed appendrel
to something else.  So we don't need them here, and omitting them from
the list should save a few cycles while processing the child rel.

Per bug #17832 from Marko Tiikkaja.

Discussion: https://postgr.es/m/17832-d0a8106cdf1b722e@postgresql.org
2023-03-12 14:20:34 -04:00
Andrew Dunstan 872e3d150e Mark unsafe_tests module as not runnable with installcheck
This was an omission in the original creation of the module.

Also slightly adjust some wording to avoid a double "is".

Backpatch the non-meson piece of this to release 12, where the module
was introduced.

Discussion: https://postgr.es/m/be869e1c-8e3f-4cde-8609-212c899cccf9@dunslane.net
2023-03-12 09:00:32 -04:00
Andres Freund a4f23f9b3c pg_amcheck: Minor test speedups
Freezing the relation N times and fetching the tuples one-by-one isn't that
cheap. On my machine this reduces test times by a bit less than one second, on
windows CI it's a few seconds.

Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/20230309001558.b7shzvio645ebdta@awork3.anarazel.de
2023-03-11 15:41:47 -08:00
Andres Freund 4f5d461e04 amcheck: Fix FullTransactionIdFromXidAndCtx() for xids before epoch 0
64bit xids can't represent xids before epoch 0 (see also be504a3e97). When
FullTransactionIdFromXidAndCtx() was passed such an xid, it'd create a 64bit
xid far into the future. Noticed while adding assertions in the course of
investigating be504a3e97, as amcheck's test create such xids.

To fix the issue, just return FirstNormalFullTransactionId in this case. A
freshly initdb'd cluster already has a newer horizon. The most minimal version
of this would make the messages for some detected corruptions differently
inaccurate. To make those cases accurate, switch
FullTransactionIdFromXidAndCtx() to use the 32bit modulo difference between
xid and nextxid to compute the 64bit xid, yielding sensible "in the future" /
"in the past" answers.

Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/20230108002923.cyoser3ttmt63bfn@awork3.anarazel.de
Backpatch: 14-, where heapam verification was introduced
2023-03-11 14:12:52 -08:00
Andres Freund 16327240da amcheck: Fix ordering bug in update_cached_xid_range()
The initialization order in update_cached_xid_range() was wrong, calling
FullTransactionIdFromXidAndCtx() before setting
->next_xid. FullTransactionIdFromXidAndCtx() uses ->next_xid.

In most situations this will not cause visible issues, because the next call
to update_cached_xid_range() will use a less wrong ->next_xid. It's rare that
xids advance fast enough for this to be a problem.

Found while adding more asserts to the 64bit xid infrastructure.

Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/20230108002923.cyoser3ttmt63bfn@awork3.anarazel.de
Backpatch: 14-, where heapam verification was introduced
2023-03-11 14:12:51 -08:00
Andres Freund 4e633016ab meson: fix header path of ossp-uuid
The ossp-uuid pkg-config file includes the necessary -I to include the header
as uuid.h. Previously this would only work if ossp-uuid has its headers in
ossp/ in an already searched path.

Reported-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/7bb8b8a5-5297-ab3c-3412-466fd0124d00@dunslane.net
2023-03-11 13:24:40 -08:00
Tom Lane 1925ac281a Fix misbehavior in contrib/pg_trgm with an unsatisfiable regex.
If the regex compiler can see that a regex is unsatisfiable
(for example, '$foo') then it may emit an NFA having no arcs.
pg_trgm's packGraph function did the wrong thing in this case;
it would access off the end of a work array, and with bad luck
could produce a corrupted output data structure causing more
problems later.  This could end with wrong answers or crashes
in queries using a pg_trgm GIN or GiST index with such a regex.

Fix by not trying to de-duplicate if there aren't at least 2 arcs.

Per bug #17830 from Alexander Lakhin.  Back-patch to all supported
branches.

Discussion: https://postgr.es/m/17830-57ff5f89bdb02b09@postgresql.org
2023-03-11 12:15:41 -05:00
Tom Lane d66bb048c3 Ensure COPY TO on an RLS-enabled table copies no more than it should.
The COPY documentation is quite clear that "COPY relation TO" copies
rows from only the named table, not any inheritance children it may
have.  However, if you enabled row-level security on the table then
this stopped being true, because the code forgot to apply the ONLY
modifier in the "SELECT ... FROM relation" query that it constructs
in order to allow RLS predicates to be attached.  Fix that.

Report and patch by Antonin Houska (comment adjustments and test case
by me).  Back-patch to all supported branches.

Discussion: https://postgr.es/m/3472.1675251957@antos
2023-03-10 13:52:44 -05:00
Jeff Davis c45dc7ffbb initdb: derive encoding from locale for ICU; similar to libc.
Previously, the default encoding was derived from the locale when
using libc; while the default was always UTF-8 when using ICU. That
would throw an error when the locale was not compatible with UTF-8.

This commit causes initdb to derive the default encoding from the
locale for both providers. If --no-locale is specified (or if the
locale is C or POSIX), the default encoding will be UTF-8 for ICU
(because ICU does not support SQL_ASCII) and SQL_ASCII for libc.

Per buildfarm failure on system "hoverfly" related to commit
27b62377b4.

Discussion: https://postgr.es/m/d191d5841347301a8f1238f609471ddd957fc47e.camel%40j-davis.com
2023-03-10 10:51:24 -08:00
Peter Eisentraut 3e623ebc7a Fix tests for non-ICU build
missed in 0d21d4b9bc
2023-03-10 14:27:55 +01:00
Peter Eisentraut 0d21d4b9bc Add standard collation UNICODE
This adds a new predefined collation named UNICODE, which sorts by the
default Unicode collation algorithm specifications, per SQL standard.

This only works if ICU support is built.

Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://www.postgresql.org/message-id/flat/1293e382-2093-a2bf-a397-c04e8f83d3c2@enterprisedb.com
2023-03-10 13:35:43 +01:00
Michael Paquier 6ad5793a49 Include headers of archive/ in installation
These new headers have been recently added in 35739b8, but they were not
installed.  Sravan has provided the patch for configure/make, while I
have fixed the meson part.

Author: Sravan Kumar, Michael Paquier
Discussion: https://postgr.es/m/CA+=NbjguiQy-MbVqfQ-jQ=2Fcmx3Zs36OkKb-vjt28jMTG0OOg@mail.gmail.com
2023-03-10 20:08:10 +09:00
Peter Eisentraut 012ee84259 Add a test for UCS_BASIC collation 2023-03-10 11:18:08 +01:00
Peter Eisentraut cd42785974 doc: Better example for custom ICU rules
Use a more practical example, and also add some explanation.

Reported-by: Jeff Davis <pgsql@j-davis.com>
2023-03-10 09:25:03 +01:00
Peter Eisentraut 470103697a Fix incorrect format placeholders 2023-03-10 07:10:43 +01:00
Michael Paquier 9ecb134a93 pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info()
This commit reworks pg_get_wal_fpi_info() to become aware of all the
block information that can be attached to a record rather than just its
full-page writes:
- Addition of the block id as assigned by XLogRegisterBuffer(),
XLogRegisterBlock() or XLogRegisterBufData().
- Addition of the block data, as bytea, or NULL if none.  The length of
the block data can be guessed with length(), so there is no need to
store its length in a separate field.
- Addition of the full-page image length, as counted without a hole or
even compressed.
- Modification of the handling of the full-page image data.  This is
still a bytea, but it could become NULL if none is assigned to a block.
- Addition of the full-page image flags, tracking if a page is stored
with a hole, if it needs to be applied and the type of compression
applied to it, as of all the BKPIMAGE_* values in xlogrecord.h.

The information of each block is returned as one single record, with the
record's ReadRecPtr included to be able to join the block information
with the existing pg_get_wal_records_info().  Note that it is perfectly
possible for a block to hold both data and full-page image.

Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the
discussion.

This commit uses some of the work proposed by Melanie, though it has
been largely redesigned and rewritten by me.  Bharath has helped in
refining a bit the whole.

Reported-by: Melanie Plageman
Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy
Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com
2023-03-10 10:09:07 +09:00
Jeff Davis 8da2ec31ee Fix test failure caused in 27b62377b4.
Per buildfarm system "prion".
2023-03-09 15:34:41 -08:00
Tom Lane bcc704b524 Reject combining "epoch" and "infinity" with other datetime fields.
Datetime input formerly accepted combinations such as
'1995-08-06 infinity', but this seems like a clear error.
Reject any combination of regular y/m/d/h/m/s fields with
these special tokens.

Joseph Koshakow, reviewed by Keisuke Kuroda and myself

Discussion: https://postgr.es/m/CAAvxfHdm8wwXwG_FFRaJ1nTHiMWb7YXS2YKCzCt8Q0a2ZoMcHg@mail.gmail.com
2023-03-09 16:49:03 -05:00
Jeff Davis 27b62377b4 Use ICU by default at initdb time.
If the ICU locale is not specified, initialize the default collator
and retrieve the locale name from that.

Discussion: https://postgr.es/m/510d284759f6e943ce15096167760b2edcb2e700.camel@j-davis.com
Reviewed-by: Peter Eisentraut
2023-03-09 10:52:41 -08:00
Alvaro Herrera a7e584a7d6
HTML docs: Add padding to table.simplelist for more readable output
This couples with a to-be-pushed pgweb patch to synchronize the other
stylesheet under which these docs are rendered on the website.

Author: Karl Pinc <kop@karlpinc.com>
Discussion: https://postgr.es/m/20230122144246.0ff87372@slate.karlpinc.com
2023-03-09 19:47:19 +01:00
Jeff Davis 206b44bb24 Fix 9637badd9f.
Discussion: https://postgr.es/m/0a364430-266e-1e1a-d5d8-1a5273c9ddb6@dunslane.net
Reported-by: Andrew Dunstan
2023-03-09 10:26:47 -08:00