Commit Graph

59455 Commits

Author SHA1 Message Date
Michael Paquier 1b373aed20 Add callback for backend initialization in pgstats
pgstat_initialize() is currently used by the WAL stats as a code path to
take some custom actions when a backend starts.  A callback is added to
generalize the concept so as all stats kinds can do the same, for
builtin and custom kinds, if set.

Reviewed-by: Bertrand Drouvot, Kyotaro Horiguchi
Discussion: https://postgr.es/m/ZtZr1K4PLdeWclXY@paquier.xyz
2024-09-05 16:05:21 +09:00
Michael Paquier 341e9a05e7 Fix two NULL pointer dereferences when reading custom pgstats from file
There were two spots in pgstat_read_statsfile() where is was possible to
finish with a null-pointer-dereference crash for custom pgstats kinds:
- When reading stats for a fixed-numbered stats entry.
- When reading a variable stats entry with name serialization.
For both cases, these issues were reachable by starting a server after
changing shared_preload_libraries so as the stats written previously
could not be loaded.

The code is changed so as the stats are ignored in this case, like the
other code paths doing similar sanity checks.  Two WARNINGs are added to
be able to debug these issues.  A test is added for the case of
fixed-numbered stats with the module injection_points.

Oversights in 7949d95945, spotted while looking at a different report.

Discussion: https://postgr.es/m/Ztj0Jftsn4xXuXtl@paquier.xyz
2024-09-05 14:36:57 +09:00
Michael Paquier 5735521ac2 Check availability of module injection_points in TAP tests
This fixes defects with installcheck for TAP tests that expect the
module injection_points to exist in an installation, but the contents of
src/test/modules are not installed by default with installcheck.  This
would cause, for example, failures under installcheck-world for a build
with injection points enabled, when the contents of src/test/modules/
are not installed.

The availability of the module can be done with a scan of
pg_available_extension.  This has been introduced in 2cdcae9da6, and
it is refactored here as a new routine in Cluster.pm.

Tests are changed in different ways depending on what they need:
- The libpq TAP test sets up a node even without injection points, so it
is enough to check that CREATE EXTENSION can be used.  There is no need
for the variable enable_injection_points.
- In test_misc, 006_signal_autovacuum requires a runtime check.
- 041_checkpoint_at_promote in recovery tests and 005_timeouts in
test_misc are updated to use the routine introduced in Cluster.pm.
- test_slru's 001_multixact, injection_points's 001_stats and
modules/gin/ do not require a check as these modules disable
installcheck entirely.

Discussion: https://postgr.es/m/ZtesYQ-WupeAK7xK@paquier.xyz
2024-09-05 13:29:43 +09:00
David Rowley 908a968612 Optimize WindowAgg's use of tuplestores
When WindowAgg finished one partition of a PARTITION BY, it previously
would call tuplestore_end() to purge all the stored tuples before again
calling tuplestore_begin_heap() and carefully setting up all of the
tuplestore read pointers exactly as required for the given frameOptions.
Since the frameOptions don't change between partitions, this part does
not make much sense.  For queries that had very few rows per partition,
the overhead of this was very large.

It seems much better to create the tuplestore and the read pointers once
and simply call tuplestore_clear() at the end of each partition.
tuplestore_clear() moves all of the read pointers back to the start
position and deletes all the previously stored tuples.

A simple test query with 1 million partitions and 1 tuple per partition
has been shown to run around 40% faster than without this change.  The
additional effort seems to have mostly been spent in malloc/free.

Making this work required adding a new bool field to WindowAggState
which had the unfortunate effect of being the 9th bool field in a group
resulting in the struct being enlarged.  Here we shuffle the fields
around a little so that the two bool fields for runcondition relating
stuff fit into existing padding.  Also, move the "runcondition" field to
be near those.  This frees up enough space with the other bool fields so
that the newly added one fits into the padding bytes.  This was done to
address a very small but apparent performance regression with queries
containing a large number of rows per partition.

Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Tatsuo Ishii <ishii@postgresql.org>
Discussion: https://postgr.es/m/CAHoyFK9n-QCXKTUWT_xxtXninSMEv%2BgbJN66-y6prM3f4WkEHw%40mail.gmail.com
2024-09-05 16:18:30 +12:00
David Rowley 19b861f880 Speedup WindowAgg code by moving uncommon code out-of-line
The code to calculate the frame offsets is only performed once per scan.
Moving this code out of line gives a small (around 4-5%) speedup when testing
with some CPUs.  Other tested CPUs are indifferent to the change.

Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Tatsuo Ishii <ishii@postgresql.org>
Discussion: https://postgr.es/m/CAApHDvqPgFtwme2Zyf75BpMLwYr2mnUstDyPiP%3DEpudYuQTPPQ%40mail.gmail.com
2024-09-05 15:59:47 +12:00
Jeff Davis 06421b0843 Remove lc_collate_is_c().
Instead just look up the collation and check collate_is_c field.

Author: Andreas Karlsson
Discussion: https://postgr.es/m/60929555-4709-40a7-b136-bcb44cff5a3c@proxel.se
2024-09-04 14:35:25 -07:00
Tom Lane 83eb481d52 Remove test-case workarounds for ancient libedit versions.
This reverts some hacks added in d33a81203 and cd69ec66c.
At the time the concern was the already-ancient version of
libedit shipped in Debian 10 (Buster).  That platform is
now two years past EOL, so desupporting it for PG 18 seems
fine.  (Also, if anyone is really hot to keep testing it,
they can use SKIP_READLINE_TESTS to skip this test.)

We might have to reconsider if any animals still in the
buildfarm don't like this, but the best way to find out
is to try it.

Anton Melnikov

Discussion: https://postgr.es/m/CAGRrpzZU48F2oV3d8eDLr=4TU9xFH5Jt9ED+qU1+X91gMH68Sw@mail.gmail.com
2024-09-04 16:25:28 -04:00
Noah Misch ddfc556a64 Revert "Optimize pg_visibility with read streams."
This reverts commit ed1b1ee59f and its
followup 1c61fd8b52.  They rendered
collect_corrupt_items() unable to detect corruption.

Discussion: https://postgr.es/m/CAN55FZ1_Ru3XpMgTwsU67FTH2fs_FrRROmb7x6zs+F44QBEiww@mail.gmail.com
Discussion: https://postgr.es/m/CAEudQAozv3wTY5TV2t29JcwPydbmKbiWQkZD42S2OgzdixPMDQ@mail.gmail.com
2024-09-04 11:36:40 -07:00
Peter Eisentraut 82b07eba9e Remove a couple of strerror() calls
Change to using %m in the error message string.  We need to be a bit
careful here to preserve errno until we need to print it.

This change avoids the use of not-thread-safe strerror() and unifies
some error message strings, and maybe makes the code appear more
consistent.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/daa87d79-c044-46c4-8458-8d77241ed7b0%40eisentraut.org
2024-09-04 14:45:31 +02:00
Michael Paquier a68159ff2b Unify some error messages to ease work of translators
This commit updates a couple of error messages around control file data,
GUCs and server settings, unifying to the same message where possible.
This reduces the translation burden a bit.

Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pv-kSN8SkxSdoHano_wPubqcg5789ejhCDZAcLFceBR-w@mail.gmail.com
2024-09-04 14:53:18 +09:00
Michael Paquier b4db64270e Apply more quoting to GUC names in messages
This is a continuation of 17974ec259.  More quotes are applied to
GUC names in error messages and hints, taking care of what seems to be
all the remaining holes currently in the tree for the GUCs.

Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pv-kSN8SkxSdoHano_wPubqcg5789ejhCDZAcLFceBR-w@mail.gmail.com
2024-09-04 13:50:44 +09:00
Amit Kapila 6c2b5edecc Collect statistics about conflicts in logical replication.
This commit adds columns in view pg_stat_subscription_stats to show the
number of times a particular conflict type has occurred during the
application of logical replication changes. The following columns are
added:

confl_insert_exists:
        Number of times a row insertion violated a NOT DEFERRABLE unique
        constraint.
confl_update_origin_differs:
        Number of times an update was performed on a row that was
        previously modified by another origin.
confl_update_exists:
        Number of times that the updated value of a row violates a
        NOT DEFERRABLE unique constraint.
confl_update_missing:
        Number of times that the tuple to be updated is missing.
confl_delete_origin_differs:
        Number of times a delete was performed on a row that was
        previously modified by another origin.
confl_delete_missing:
        Number of times that the tuple to be deleted is missing.

The update_origin_differs and delete_origin_differs conflicts can be
detected only when track_commit_timestamp is enabled.

Author: Hou Zhijie
Reviewed-by: Shveta Malik, Peter Smith, Anit Kapila
Discussion: https://postgr.es/m/OS0PR01MB57160A07BD575773045FC214948F2@OS0PR01MB5716.jpnprd01.prod.outlook.com
2024-09-04 08:55:21 +05:30
Richard Guo 9626068f13 Avoid unnecessary post-sort projection
When generating paths for the ORDER BY clause, one thing we need to
ensure is that the output paths project the correct final_target.  To
achieve this, in create_ordered_paths, we compare the pathtarget of
each generated path with the given 'target', and add a post-sort
projection step if the two targets do not match.

Currently we perform a simple pointer comparison between the two
targets.  It turns out that this is not sufficient.  Each sorted_path
generated in create_ordered_paths initially projects the correct
target required by the preceding steps of sort.  If it is the same
pointer as sort_input_target, pointer comparison suffices, because
sort_input_target is always identical to final_target when no
post-sort projection is needed.

However, sorted_path's initial pathtarget may not be the same pointer
as sort_input_target, because in apply_scanjoin_target_to_paths, if
the target to be applied has the same expressions as the existing
reltarget, we only inject the sortgroupref info into the existing
pathtargets, rather than create new projection paths.  As a result,
pointer comparison in create_ordered_paths is not reliable.

Instead, we can compare PathTarget.exprs to determine whether a
projection step is needed.  If the expressions match, we can be
confident that a post-sort projection is not required.

It could be argued that this change adds extra check cost each time we
decide whether a post-sort projection is needed.  However, as
explained in apply_scanjoin_target_to_paths, by avoiding the creation
of projection paths, we save effort both immediately and at plan
creation time.  This, I think, justifies the extra check cost.

There are two ensuing plan changes in the regression tests, but they
look reasonable and are exactly what we are fixing here.  So no
additional test cases are added.

No backpatch as this could result in plan changes.

Author: Richard Guo
Reviewed-by: Peter Eisentraut, David Rowley, Tom Lane
Discussion: https://postgr.es/m/CAMbWs48TosSvmnz88663_2yg3hfeOFss-J2PtnENDH6J_rLnRQ@mail.gmail.com
2024-09-04 12:19:19 +09:00
Richard Guo 4f1124548f Check the validity of commutators for merge/hash clauses
When creating merge or hash join plans in createplan.c, the merge or
hash clauses may need to get commuted to ensure that the outer var is
on the left and the inner var is on the right if they are not already
in the expected form.  This requires that their operators have
commutators.  Failing to find a commutator at this stage would result
in 'ERROR: could not find commutator for operator xxx', with no
opportunity to select an alternative plan.

Typically, this is not an issue because mergejoinable or hashable
operators are expected to always have valid commutators.  But in some
artificial cases this assumption may not hold true.  Therefore, here
in this patch we check the validity of commutators for clauses in the
form "inner op outer" when selecting mergejoin/hash clauses, and
consider a clause unusable for the current pair of outer and inner
relations if it lacks a commutator.

There are not (and should not be) any such operators built into
Postgres that are mergejoinable or hashable but have no commutators;
so we leverage the alias type 'int8alias1' created in equivclass.sql
to build the test case.  This is why the test case is included in
equivclass.sql rather than in join.sql.

Although this is arguably a bug fix, it cannot be reproduced without
installing an incomplete opclass, which is unlikely to happen in
practice, so no back-patch.

Reported-by: Alexander Pyhalov
Author: Richard Guo
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/c59ec04a2fef94d9ffc35a9b17dfc081@postgrespro.ru
2024-09-04 12:17:11 +09:00
Michael Paquier 08b9b9e043 Fix inconsistent LWLock tranche name "CommitTsSLRU"
This term was using an inconsistent casing between the code and the
documentation, using "CommitTsSLRU" in wait_event_names.txt and
"CommitTSSLRU" in the code.

Let's update the term in the code to reflect what's in the
documentation, "CommitTs" being more commonly used, so as
pg_stat_activity shows the same term as the documentation.

Oversight in 53c2a97a92.

Author: Alexander Lakhin
Discussion: https://postgr.es/m/f7e514cf-2446-21f1-a5d2-8c089a6e2168@gmail.com
Backpatch-through: 17
2024-09-04 10:21:17 +09:00
Michael Paquier 2cdcae9da6 Avoid installcheck failure in TAP tests using injection_points
These tests depend on the test module injection_points to be installed,
but it may not be available as the contents of src/test/modules/ are not
installed by default.

This commit adds a workaround based on a scan of pg_available_extensions
to check if the extension is available, skipping the test if it is not.
This allows installcheck to work transparently.

There are more tests impacted by this problem on HEAD, but for now this
addresses only the tests that exist on HEAD and v17 as the release is
close by.

Reported-by: Maxim Orlov
Discussion: https://postgr.es/m/CACG=ezZkoT-pFz6a9XnyToiuR-Wg8fGELqHLoyBodr+2h-77qA@mail.gmail.com
Backpatch-through: 17
2024-09-04 08:56:23 +09:00
Jeff Davis 12d3345c0d Remember last collation to speed up collation cache.
This optimization is to avoid a performance regression in an upcoming
patch that will remove lc_collate_is_c().

Discussion: https://postgr.es/m/96a559be83329bc66074a3925ebcfa8ceb16dfc5.camel@j-davis.com
Discussion: https://postgr.es/m/646f662e145ab38cff1c04d475f4448f53fc5042.camel@j-davis.com
Discussion: https://postgr.es/m/54565933-d82f-4d7c-8f47-288b1b570fd8@eisentraut.org
2024-09-03 16:30:03 -07:00
Michael Paquier 516ff05539 Simplify makefiles exporting twice enable_injection_points
This is confusing, as it exports twice the same variable.  Oversight in
6782709df8 that has spread in more places afterwards.

Reported-by: Alvaro Herrera, Tom Lane
Discussion: https://postgr.es/m/202408201630.mn6vbohjh7hh@alvherre.pgsql
Backpatch-through: 17
2024-09-04 08:05:44 +09:00
Thomas Munro 813fde73d4 Standardize "read-ahead advice" terminology.
Commit 6654bb920 added macOS's equivalent of POSIX_FADV_WILLNEED, and
changed some explicit references to posix_fadvise to use this more
general name for the concept.  Update some remaining references.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/0827edec-1317-4917-a186-035eb1e3241d%40eisentraut.org
2024-09-04 10:28:53 +12:00
Noah Misch 1c61fd8b52 Fix stack variable scope from previous commit.
The defect came from me, not from that commit's credited author.  Per
buildfarm members olingo and grassquit.

Discussion: https://postgr.es/m/20240903192030.1e@rfd.leadboat.com
2024-09-03 12:44:54 -07:00
Noah Misch ed1b1ee59f Optimize pg_visibility with read streams.
We've measured 5% performance improvement, and this arranges to benefit
automatically from future optimizations to the read_stream subsystem.

Nazir Bilal Yavuz

Discussion: https://postgr.es/m/CAN55FZ1_Ru3XpMgTwsU67FTH2fs_FrRROmb7x6zs+F44QBEiww@mail.gmail.com
2024-09-03 10:46:20 -07:00
Noah Misch c582b75851 Add block_range_read_stream_cb(), to deduplicate code.
This replaces two functions for iterating over all blocks in a range.  A
pending patch will use this instead of adding a third.

Nazir Bilal Yavuz

Discussion: https://postgr.es/m/20240820184742.f2.nmisch@google.com
2024-09-03 10:46:20 -07:00
Daniel Gustafsson ba7625a7a5 Use library functions to edit config in SSL tests
The SSL tests were editing the postgres configuration by directly
reading and writing the files rather than using append_conf() from
the testcode library.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/01F4684C-8C98-4BBE-AB83-AC8D7C746AF8@yesql.se
2024-09-03 18:57:56 +02:00
Daniel Gustafsson e5f1f0a4f2 Test for PG_TEST_EXTRA separately in SSL tests
PG_TEST_EXTRA is an override and should be tested for separately
from any other test as there is no dependency on whether OpenSSL
is available or not.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/01F4684C-8C98-4BBE-AB83-AC8D7C746AF8@yesql.se
2024-09-03 18:57:54 +02:00
Daniel Gustafsson 31a98934d1 Fix typos in code comments and test data
The typos in 005_negotiate_encryption.pl and pg_combinebackup.c
shall be backported to v17 where they were introduced.

Backpatch-through: v17
Discussion: https://postgr.es/m/Ztaj7BkN4658OMxF@paquier.xyz
2024-09-03 11:33:38 +02:00
Peter Eisentraut 2b5f57977f Add const qualifiers to XLogRegister*() functions
Add const qualifiers to XLogRegisterData() and XLogRegisterBufData().
Several unconstify() calls can be removed.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/dd889784-9ce7-436a-b4f1-52e4a5e577bd@eisentraut.org
2024-09-03 08:06:03 +02:00
Michael Paquier 4236825197 Fix typos and grammar in code comments and docs
Author: Alexander Lakhin
Discussion: https://postgr.es/m/f7e514cf-2446-21f1-a5d2-8c089a6e2168@gmail.com
2024-09-03 14:49:04 +09:00
Michael Paquier c7cd2d6ed0 Define PG_TBLSPC_DIR for path pg_tblspc/ in data folder
Similarly to 2065ddf5e3, this introduces a define for "pg_tblspc".
This makes the style more consistent with the existing PG_STAT_TMP_DIR,
for example.

There is a difference with the other cases with the introduction of
PG_TBLSPC_DIR_SLASH, required in two places for recovery and backups.

Author: Bertrand Drouvot
Reviewed-by: Ashutosh Bapat, Álvaro Herrera, Yugo Nagata, Michael
Paquier
Discussion: https://postgr.es/m/ZryVvjqS9SnV1GPP@ip-10-97-1-34.eu-west-3.compute.internal
2024-09-03 09:11:54 +09:00
Daniel Gustafsson 94eec79633 doc: Consistently use result set in documentation
We use "result set" in all other places so let's be consistent
across the entire documentation.

Reported-by: grantgryczan@gmail.com
Discussion: https://postgr.es/m/172187924855.915373.15595156724215203822@wrigleys.postgresql.org
2024-09-02 18:36:57 +02:00
Peter Eisentraut 2befd22790 Fix rarely-run test for message wording change
fixup for 2e6a8047f0

Reported-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
2024-09-02 17:40:32 +02:00
Daniel Gustafsson c3333dbc0c Only perform pg_strong_random init when required
The random number generator in OpenSSL 1.1.1 was redesigned to provide
fork safety by default, thus removing the need for calling RAND_poll
after forking to ensure that two processes cannot share the same state.
Since we now support 1.1.0 as the minumum version, and 1.1.0 is being
increasingly phased out from production use, only perform the RAND_poll
initialization for installations running 1.1.0 by checking the OpenSSL
version number.

LibreSSL changed random number generator when forking OpenSSL and has
provided fork safety since version 2.0.2.

This removes the overhead of initializing the RNG for strong random
for the vast majority of users for whom it is no longer required.

Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA+hUKGKh7QrYzu=8yWEUJvXtMVm_CNWH1L_TLWCbZMwbi1XP2Q@mail.gmail.com
2024-09-02 13:52:27 +02:00
Daniel Gustafsson a70e01d430 Remove support for OpenSSL older than 1.1.0
OpenSSL 1.0.2 has been EOL from the upstream OpenSSL project for
some time, and is no longer the default OpenSSL version with any
vendor which package PostgreSQL. By retiring support for OpenSSL
1.0.2 we can remove a lot of no longer required complexity for
managing state within libcrypto which is now handled by OpenSSL.

Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/ZG3JNursG69dz1lr@paquier.xyz
Discussion: https://postgr.es/m/CA+hUKGKh7QrYzu=8yWEUJvXtMVm_CNWH1L_TLWCbZMwbi1XP2Q@mail.gmail.com
2024-09-02 13:51:48 +02:00
Daniel Gustafsson 6ebeeae296 Cache typarray for fast lookups in binary upgrade mode
When upgrading a large schema it adds significant overhead to perform
individual catalog lookups per relation in order to retrieve Oid for
preserving Oid calls. This instead adds the typarray to the TypeInfo
cache which then allows for fast lookups using the existing API. A
35% reduction of pg_dump runtime in binary upgrade mode was observed
with this change.

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/8F1F1E1D-D17B-4B33-B014-EDBCD15F3F0B@yesql.se
2024-09-02 10:17:46 +02:00
Peter Eisentraut 4d5111b3f1 More use of getpwuid_r() directly
Remove src/port/user.c, call getpwuid_r() directly.  This reduces some
complexity and allows better control of the error behavior.  For
example, the old code would in some circumstances silently truncate
the result string, or produce error message strings that the caller
wouldn't use.

src/port/user.c used to be called src/port/thread.c and contained
various portability complications to support thread-safety.  These are
all obsolete, and all but the user-lookup functions have already been
removed.  This patch completes this by also removing the user-lookup
functions.

Also convert src/backend/libpq/auth.c to use getpwuid_r() for
thread-safety.

Originally, I tried to be overly correct by using
sysconf(_SC_GETPW_R_SIZE_MAX) to get the buffer size for getpwuid_r(),
but that doesn't work on FreeBSD.  All the OS where I could find the
source code internally use 1024 as the suggested buffer size, so I
just ended up hardcoding that.  The previous code used BUFSIZ, which
is an unrelated constant from stdio.h, so its use seemed
inappropriate.

Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/5f293da9-ceb4-4937-8e52-82c25db8e4d3%40eisentraut.org
2024-09-02 09:04:30 +02:00
Michael Paquier 23138284cd Rename enum labels of PG_Locale_Strategy
PG_REGEX_BUILTIN was added in f69319f2f1 but it did not follow the
same pattern as the previous labels, i.e. PG_LOCALE_*.  In addition to
this, the two libc strategies did not include in the name that they were
related to this library.

The enum labels are renamed as PG_STRATEGY_type[_subtype] to make the
code clearer, in accordance to the library and the functions they rely
on.

Author: Andreas Karlsson
Discussion: https://postgr.es/m/6f81200f-68fd-411e-97a1-d1f291d2e222@proxel.se
2024-09-02 08:18:41 +09:00
Thomas Munro 4effd0844d Fix unfairness in all-cached parallel seq scan.
Commit b5a9b18c introduced block streaming infrastructure with a special
fast path for all-cached scans, and commit b7b0f3f2 connected the
infrastructure up to sequential scans.  One of the fast path
micro-optimizations had an unintended consequence: it interfered with
parallel sequential scan's block range allocator (from commit 56788d21),
which has its own ramp-up and ramp-down algorithm when handing out
groups of pages to workers.  A scan of an all-cached table could give
extra blocks to one worker, when others had finished.  In some plans
(probably already very bad plans, such as the one reported by
Alexander), the unfairness could be magnified.

An internal buffer of 16 block numbers is removed, keeping just a single
block buffer for technical reasons.

Back-patch to 17.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/63a63690-dd92-c809-0b47-af05459e95d1%40gmail.com
2024-08-31 17:28:02 +12:00
Thomas Munro ecd56459cf Stabilize 039_end_of_wal test.
The first test was sensitive to the insert LSN after setting up the
catalogs, which depended on environmental things like the locales on the
OS and usernames.  Switch to a new WAL file before the first test, as a
simple way to put every computer into the same state.

Back-patch to all supported releases.

Reported-by: Anton Voloshin <a.voloshin@postgrespro.ru>
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/b26aeac2-cb6d-4633-a7ea-945baae83dcf%40postgrespro.ru
2024-08-31 14:48:44 +12:00
Masahiko Sawada d7613ea72f Clarify restrict_nonsystem_relation_kind description.
This change improves the description of the
restrict_nonsystem_relation_kind parameter in guc_table.c and the
documentation for better clarity.

Backpatch to 12, where this GUC parameter was introduced.

Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/6a96f1af-22b4-4a80-8161-1f26606b9ee2%40eisentraut.org
Backpatch-through: 12
2024-08-30 15:06:09 -07:00
Tom Lane 0e5c823806 Make postgres_fdw's query_cancel test less flaky.
This test occasionally shows

+WARNING:  could not get result of cancel request due to timeout

which appears to be because the cancel request is sometimes unluckily
sent to the remote session between queries, and then it's ignored.

This patch tries to make that less probable in three ways:

1. Use a test query that does not involve remote estimates, so that
no EXPLAINs are sent.
2. Make sure that the remote session is ready-to-go (transaction
started, SET commands sent) before we start the timer.
3. Increase the statement_timeout to 100ms, to give the local
session enough time to plan and issue the query.

We might have to go higher than 100ms to make this adequately
stable in the buildfarm, but let's see how it goes.

Back-patch to v17 where this test was introduced.

Jelte Fennema-Nio and Tom Lane

Discussion: https://postgr.es/m/578934.1725045685@sss.pgh.pa.us
2024-08-30 16:47:39 -04:00
Tom Lane cb8e50a4a0 Avoid inserting PlaceHolderVars in cases where pre-v16 PG did not.
Commit 2489d76c4 removed some logic from pullup_replace_vars()
that avoided wrapping a PlaceHolderVar around a pulled-up
subquery output expression if the expression could be proven
to go to NULL anyway (because it contained Vars or PHVs of the
pulled-up relation and did not contain non-strict constructs).
But removing that logic turns out to cause performance regressions
in some cases, because the extra PHV blocks subexpression folding,
and will do so even if outer-join reduction later turns it into a
no-op with no phnullingrels bits.  This can for example prevent
an expression from being matched to an index.

The reason for always adding a PHV was to ensure we had someplace
to put the varnullingrels marker bits of the Var being replaced.
However, it turns out we can optimize in exactly the same cases that
the previous code did, because we can instead attach the needed
varnullingrels bits to the contained Var(s)/PHV(s).

This is not a complete solution --- it would be even better if we
could remove PHVs after reducing them to no-ops.  It doesn't look
practical to back-patch such an improvement, but this change seems
safe and at least gets rid of the performance-regression cases.

Per complaint from Nikhil Raj.  Back-patch to v16 where the
problem appeared.

Discussion: https://postgr.es/m/CAG1ps1xvnTZceKK24OUfMKLPvDP2vjT-d+F2AOCWbw_v3KeEgg@mail.gmail.com
2024-08-30 12:42:12 -04:00
Tom Lane 3409b4db63 Remove one memoize test case added by commit 069d0ff02.
This test case turns out to depend on the assumption that a non-Var
subquery output that's underneath an outer join will always get
wrapped in a PlaceHolderVar.  But that behavior causes performance
regressions in some cases compared to what happened before v16.
The next commit will avoid inserting a PHV in the same cases where
pre-v16 did, and that causes get_memoized_path to not detect that
a memoize plan could be used.

Commit this separately, in hopes that we can restore the test after
making get_memoized_path smarter.  (It's failing to find memoize
plans in adjacent cases where no PHV was ever inserted, so there
is definitely room for improvement there.)

Discussion: https://postgr.es/m/CAG1ps1xvnTZceKK24OUfMKLPvDP2vjT-d+F2AOCWbw_v3KeEgg@mail.gmail.com
2024-08-30 12:22:31 -04:00
Michael Paquier c39afc38cf Define PG_LOGICAL_DIR for path pg_logical/ in data folder
This is similar to 2065ddf5e3, but this time for pg_logical/ itself
and its contents, like the paths for snapshots, mappings or origin
checkpoints.

Author: Bertrand Drouvot
Reviewed-by: Ashutosh Bapat, Yugo Nagata, Michael Paquier
Discussion: https://postgr.es/m/ZryVvjqS9SnV1GPP@ip-10-97-1-34.eu-west-3.compute.internal
2024-08-30 15:25:12 +09:00
Michael Paquier 2065ddf5e3 Define PG_REPLSLOT_DIR for path pg_replslot/ in data folder
This commit replaces most of the hardcoded values of "pg_replslot" by a
new PG_REPLSLOT_DIR #define.  This makes the style more consistent with
the existing PG_STAT_TMP_DIR, for example.  More places will follow a
similar change.

Author: Bertrand Drouvot
Reviewed-by: Ashutosh Bapat, Yugo Nagata, Michael Paquier
Discussion: https://postgr.es/m/ZryVvjqS9SnV1GPP@ip-10-97-1-34.eu-west-3.compute.internal
2024-08-30 10:42:21 +09:00
Michael Paquier a83a944e9f Rename pg_sequence_read_tuple() to pg_get_sequence_data()
This commit removes log_cnt from the tuple returned by the SQL function.
This field is an internal counter that tracks when a WAL record should
be generated for a sequence, and it is reset each time the sequence is
restored or recovered.  It is not necessary to rebuild the sequence DDL
commands for pg_dump and pg_upgrade where this function is used.  The
field can still be queried with a scan of the "table" created
under-the-hood for a sequence.

Issue noticed while hacking on a feature that can rely on this new
function rather than pg_sequence_last_value(), aimed at making sequence
computation more easily pluggable.

Bump catalog version.

Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/Zsvka3r-y2ZoXAdH@paquier.xyz
2024-08-30 08:49:24 +09:00
Tom Lane 43f2e7634d Fix mis-deparsing of ORDER BY lists when there is a name conflict.
If an ORDER BY item in SELECT is a bare identifier, the parser
first seeks it as an output column name of the SELECT (for SQL92
compatibility).  However, ruleutils.c is expecting the SQL99
interpretation where such a name is an input column name.  So it's
possible to produce an incorrect display of a view in the (admittedly
pretty ill-advised) case where some other column is renamed in the
SELECT output list to match an ORDER BY column.

This can be fixed by table-qualifying such names in the dumped
view text.  To avoid cluttering less-ill-advised queries, we'd
like to do so only when there's an actual name conflict.
That requires passing the current get_query_def call's resultDesc
parameter down to get_variable, so that it can determine what
the output column names are.  In hopes of reducing rather than
increasing notational clutter in ruleutils.c, I moved that value
into the deparse_context struct and removed it from the parameter
lists of get_query_def's other subroutines.

I made a few other cosmetic changes while at it:
* Likewise move the colNamesVisible parameter into deparse_context.
* Rename deparse_context's windowTList field to targetList,
since it's no longer used only in connection with WINDOW clauses.
* Replace the special_exprkind field with a bool inGroupBy,
since that was all it was being used for, and the apparent
flexibility of storing a ParseExprKind proved to be illusory.
(We need a separate varInOrderBy field to make this patch work.)
* Remove useless save/restore logic in get_select_query_def.

In principle, this bug is quite old.  However, it seems unreachable
before 1b4d280ea, because before that the presence of "new" and "old"
entries in a view's rangetable caused us to always table-qualify every
Var reference in dumped views.  Hence, back-patch to v16 where that
came in.

Per bug #18589 from Quynh Tran.

Discussion: https://postgr.es/m/18589-70091cb81db1a3f1@postgresql.org
2024-08-29 13:24:17 -04:00
Peter Eisentraut edee0c621d Message style improvements 2024-08-29 14:43:34 +02:00
Peter Eisentraut 894be11adf Put generated_stored test objects in a schema
This avoids naming conflicts with concurrent tests with similarly
named objects.  Currently, there are none, but a tests for virtual
generated columns are planned to be added.

Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Tomasz Rybak <tomasz.rybak@post.pl>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
2024-08-29 12:24:47 +02:00
Peter Eisentraut b9ed496925 Rename regress test generated to generated_stored
This makes naming room to have another test file for virtual generated
columns.

Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Tomasz Rybak <tomasz.rybak@post.pl>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
2024-08-29 12:08:31 +02:00
Peter Eisentraut 4d68a04324 Disallow USING clause when altering type of generated column
This does not make sense.  It would write the output of the USING
clause into the converted column, which would violate the generation
expression.  This adds a check to error out if this is specified.

There was a test for this, but that test errored out for a different
reason, so it was not effective.

Reported-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Yugo NAGATA <nagata@sraoss.co.jp>
Discussion: https://www.postgresql.org/message-id/flat/c7083982-69f4-4b14-8315-f9ddb20b9834%40eisentraut.org
2024-08-29 09:06:15 +02:00
Heikki Linnakangas 478846e768 Rename some shared memory initialization routines
To make them follow the usual naming convention where
FoobarShmemSize() calculates the amount of shared memory needed by
Foobar subsystem, and FoobarShmemInit() performs the initialization.

I didn't rename CreateLWLocks() and InitShmmeIndex(), because they are
a little special. They need to be called before any of the other
ShmemInit() functions, because they set up the shared memory
bookkeeping itself. I also didn't rename InitProcGlobal(), because
unlike other Shmeminit functions, it's not called by individual
backends.

Reviewed-by: Andreas Karlsson
Discussion: https://www.postgresql.org/message-id/c09694ff-2453-47e5-b26c-32a16cd75ce6@iki.fi
2024-08-29 09:46:21 +03:00