59149 Commits

Author SHA1 Message Date
Nathan Bossart
64f34eb2e2 Use CREATE DATABASE ... STRATEGY = FILE_COPY in pg_upgrade.
While this strategy is ordinarily quite costly because it requires
performing two checkpoints, testing shows that it tends to be a
faster choice than WAL_LOG during pg_upgrade, presumably because
fsync is turned off.  Furthermore, we can skip the checkpoints
altogether because the problems they are intended to prevent don't
apply to pg_upgrade.  Instead, we just need to CHECKPOINT once in
the new cluster after making any changes to template0 and before
restoring the rest of the databases.  This ensures that said
template0 changes are written out to disk prior to creating the
databases via FILE_COPY.

Co-authored-by: Matthias van de Meent
Reviewed-by: Ranier Vilela, Dilip Kumar, Robert Haas, Michael Paquier
Discussion: https://postgr.es/m/Zl9ta3FtgdjizkJ5%40nathan
2024-07-08 16:18:00 -05:00
Andrew Dunstan
4b4b931bcd Choose ports for test servers less likely to result in conflicts
If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com

Backpatch to all live branches (12 and up)
2024-07-08 11:40:58 -04:00
Andrew Dunstan
e4f4c5424c Force nodes for SSL tests to start in TCP mode
Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com

Backpatch to all live branches (12 and up).
2024-07-08 11:40:58 -04:00
Tom Lane
6082b3d5d3 Use xmlParseInNodeContext not xmlParseBalancedChunkMemory.
xmlParseInNodeContext has basically the same functionality with
a different API: we have to supply an xmlNode that's attached to a
document rather than just the document.  That's not hard though.
The benefits are two:

* Early 2.13.x releases of libxml2 contain a bug that causes
xmlParseBalancedChunkMemory to return the wrong status value in some
cases.  This breaks our regression tests.  While that bug is now fixed
upstream and will probably never be seen in any production-oriented
distro, it is currently a problem on some more-bleeding-edge-friendly
platforms.

* xmlParseBalancedChunkMemory is considered to depend on libxml2's
semi-deprecated SAX1 APIs, and will go away when and if they do.
There may already be libxml2 builds out there that lack this function.

So there are both short- and long-term reasons to make this change.

While here, avoid allocating an xmlParserCtxt in DOCUMENT parse mode,
since that code path is not going to use it.

Like 066e8ac6e, this will need to be back-patched.  This is just a
trial commit to see if the buildfarm agrees that we can use
xmlParseInNodeContext unconditionally.

Erik Wienhold and Tom Lane, per report from Frank Streitzig.

Discussion: https://postgr.es/m/trinity-b0161630-d230-4598-9ebc-7a23acdb37cb-1720186432160@3c-app-gmx-bap25
Discussion: https://postgr.es/m/trinity-361ba18b-541a-4fe7-bc63-655ae3a7d599-1720259822452@3c-app-gmx-bs01
2024-07-08 14:04:00 -04:00
Dean Rasheed
1ff39f4ff2 Fix scale clamping in numeric round() and trunc().
The numeric round() and trunc() functions clamp the scale argument to
the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much
smaller than the actual allowed range of type numeric. As a result,
they return incorrect results when asked to round/truncate more than
2000 digits before or after the decimal point.

Fix by using the correct upper and lower scale limits based on the
actual allowed (and documented) range of type numeric.

While at it, use the new NUMERIC_WEIGHT_MAX constant instead of
SHRT_MAX in all other overflow checks, and fix a comment thinko in
power_var() introduced by e54a758d24 -- the minimum value of
ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this
doesn't affect the point being made in the comment, that the resulting
local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000).

Back-patch to all supported branches.

Dean Rasheed, reviewed by Joel Jacobson.

Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com
2024-07-08 17:48:45 +01:00
Amit Langote
519d710720 Typo fix
Reported-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/CAEG8a3KPi=LayiTwJ11ikF7bcqnZUrcj8NgX0V8nO1mQKZ9GfQ@mail.gmail.com
Backpatch-through: 17
2024-07-08 22:12:55 +09:00
Heikki Linnakangas
cc68ca6d42 Fix outdated comment after removal of direct SSL fallback
The option to fall back from direct SSL to negotiated SSL or a
plaintext connection was removed in commit fb5718f35f.

Discussion: https://www.postgresql.org/message-id/c82ad227-e049-4e18-8898-475a748b5a5a@iki.fi
2024-07-08 12:44:45 +03:00
Michael Paquier
e311c6e539 Renumber pg_get_acl() in pg_proc.dat
a6417078c414 has introduced as project policy that new features
committed during the development cycle should use new OIDs in the
[8000,9999] range.

4564f1cebd43 did not respect that rule, so let's renumber pg_get_acl()
to use an OID in the correct range.

Bump catalog version.
2024-07-08 15:34:33 +09:00
David Rowley
7340d9362a Widen lossy and exact page counters for Bitmap Heap Scan
Both of these counters were using the "long" data type.  On MSVC that's
a 32-bit type.  On modern hardware, I was able to demonstrate that we can
wrap those counters with a query that only takes 15 minutes to run.

This issue may manifest itself either by not showing the values of the
counters because they've wrapped and are less than zero, resulting in
them being filtered by the > 0 checks in show_tidbitmap_info(), or bogus
numbers being displayed which are modulus 2^32 of the actual number.

Widen these counters to uint64.

Discussion: https://postgr.es/m/CAApHDvpS_97TU+jWPc=T83WPp7vJa1dTw3mojEtAVEZOWh9bjQ@mail.gmail.com
2024-07-08 14:43:09 +12:00
Richard Guo
d7db04dfda Remove an extra period in code comment
Author: Junwang Zhao
Discussion: https://postgr.es/m/CAEG8a3L9GgfKc+XT+NMHPY7atAOVYqjUqKEFQKhcPHFYRW=PuQ@mail.gmail.com
2024-07-08 11:17:22 +09:00
Richard Guo
0ffc0acaf3 Fix right-anti-joins when the inner relation is proven unique
For an inner_unique join, we always assume that the executor will stop
scanning for matches after the first match.  Therefore, for a mergejoin
that is inner_unique and whose mergeclauses are sufficient to identify a
match, we set the skip_mark_restore flag to true, indicating that the
executor need not do mark/restore calls.  However, merge-right-anti-join
did not get this memo and continues scanning the inner side for matches
after the first match.  If there are duplicates in the outer scan, we
may incorrectly skip matching some inner tuples, which can lead to wrong
results.

Here we fix this issue by ensuring that merge-right-anti-join also
advances to next outer tuple after the first match in inner_unique
cases.  This also saves cycles by avoiding unnecessary scanning of inner
tuples after the first match.

Although hash-right-anti-join does not suffer from this wrong results
issue, we apply the same change to it as well, to help save cycles for
the same reason.

Per bug #18522 from Antti Lampinen, and bug #18526 from Feliphe Pozzer.
Back-patch to v16 where right-anti-join was introduced.

Author: Richard Guo
Discussion: https://postgr.es/m/18522-c7a8956126afdfd0@postgresql.org
2024-07-08 10:11:46 +09:00
Michael Paquier
74b8e6a698 Re-enable autoruns for cmd.exe on Windows
This acts as a revert of b83747a8a65b and 9886744a361b.  As pointed out
by Noah, HEAD and REL_17_STABLE are in a weird state where the code
paths adding /D would limit the spawn of child processes, but we still
have code paths where the spawn of more than one child process(es) would
be possible.

Let's remove these /D switches for now, to bring back the code into a
state consistent with how autorun is configured on a Windows host.

Reported-by: Noah Misch
Discussion: https://postgr.es/m/20240630021211.f3.nmisch@google.com
Backpatch-through: 17
2024-07-08 09:43:59 +09:00
Tom Lane
066e8ac6ea Use xmlAddChildList not xmlAddChild in XMLSERIALIZE.
It looks like we should have been doing this all along,
but we got away with the wrong coding until libxml2 2.13.0
tightened up xmlAddChild's behavior.

There is more stuff to be fixed to be compatible with 2.13.0,
and it will all need to be back-patched.  This is just a
trial commit to see if the buildfarm agrees that we can use
xmlAddChildList unconditionally.

Erik Wienhold, per report from Frank Streitzig.

Discussion: https://postgr.es/m/trinity-b0161630-d230-4598-9ebc-7a23acdb37cb-1720186432160@3c-app-gmx-bap25
Discussion: https://postgr.es/m/trinity-361ba18b-541a-4fe7-bc63-655ae3a7d599-1720259822452@3c-app-gmx-bs01
2024-07-06 15:16:13 -04:00
David Rowley
04bcf9e19a Adjust tuplestore.c not to allocate BufFiles in generation context
590b045c3 made it so tuplestore.c would store tuples inside a
generation.c memory context.  After fixing a bug report in 97651b013, it
seems that it's probably best not to allocate BufFile related
allocations in that context.  Let's keep it just for tuple data.

This adjusts the code to switch to the Tuplestorestate.context's parent,
which is the MemoryContext that tuplestore_begin_common() was called in.
It does not seem worth adding a new field in Tuplestorestate to store
this when we can access it by looking at the Tuplestorestate's
context's parent.

Discussion: https://postgr.es/m/CAApHDvqFt_CdJtSr+E9YLZb7jZAyRCy3hjQ+ktM+dcOFVq-xkg@mail.gmail.com
2024-07-06 17:40:05 +12:00
David Rowley
97651b0139 Fix incorrect sentinel byte logic in GenerationRealloc()
This only affects MEMORY_CONTEXT_CHECKING builds.

This fixes an off-by-one issue in GenerationRealloc() where the
fast-path code which tries to reuse the existing allocation if the
existing chunk is >= the new requested size.  The code there thought it
was always ok to use the existing chunk, but when oldsize == size there
isn't enough space to store the sentinel byte.  If both sizes matched
exactly set_sentinel() would overwrite the first byte beyond the chunk
and then subsequent GenerationRealloc() calls could then fail the
Assert(chunk->requested_size < oldsize) check which is trying to ensure
the chunk is large enough to store the sentinel.

The same issue does not exist in aset.c as the sentinel checking code
only adds a sentinel byte if there's enough space in the chunk.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/49275921-7b39-41af-5eb8-97b50ce3312e@gmail.com
Backpatch-through: 16, where the problem was introduced by 0e480385e
2024-07-06 13:59:34 +12:00
Thomas Munro
2a5ef09830 Cope with <regex.h> name clashes.
macOS 15's SDK pulls in headers related to <regex.h> when we include
<xlocale.h>.  This causes our own regex_t implementation to clash with
the OS's regex_t implementation.  Luckily our function names already had
pg_ prefixes, but the macros and typenames did not.

Include <regex.h> explicitly on all POSIX systems, and fix everything
that breaks.  Then we can prove that we are capable of fully hiding and
replacing the system regex API with our own.

1.  Deal with standard-clobbering macros by undefining them all first.
POSIX says they are "symbolic constants".  If they are macros, this
allows us to redefine them.  If they are enums or variables, our macros
will hide them.

2.  Deal with standard-clobbering types by giving our types pg_
prefixes, and then using macros to redirect xxx_t -> pg_xxx_t.

After including our "regex/regex.h", the system <regex.h> is hidden,
because we've replaced all the standard names.  The PostgreSQL source
tree and extensions can continue to use standard prefix-less type and
macro names, but reach our implementation, if they included our
"regex/regex.h" header.

Back-patch to all supported branches, so that macOS 15's tool chain can
build them.

Reported-by: Stan Hu <stanhu@gmail.com>
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Tested-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/CAMBWrQnEwEJtgOv7EUNsXmFw2Ub4p5P%2B5QTBEgYwiyjy7rAsEQ%40mail.gmail.com
2024-07-06 10:27:16 +12:00
Tom Lane
8212625e53 Fix placement of "static".
Various buildfarm critters were complaining about

pgbench.c:304:1: warning: 'static' is not at beginning of declaration [-Wold-style-declaration]

Evidently a thinko in 720b0eaae.
2024-07-05 17:32:55 -04:00
Nathan Bossart
0b1fe1413e Remove check hooks for GUCs that contribute to MaxBackends.
Each of max_connections, max_worker_processes,
autovacuum_max_workers, and max_wal_senders has a GUC check hook
that verifies the sum of those GUCs does not exceed a hard-coded
limit (see the comment for MAX_BACKENDS in postmaster.h).  In
general, the hooks effectively guard against egregious
misconfigurations.

However, this approach has some problems.  Since these check hooks
are called as each GUC is assigned its user-specified value, only
one of the hooks will be called with all the relevant GUCs set.  If
one or more of the user-specified values are less than the initial
values of the GUCs' underlying variables, false positives can
occur.

Furthermore, the error message emitted when one of the check hooks
fails is not tremendously helpful.  For example, the command

	$ pg_ctl -D . start -o "-c max_connections=262100 -c max_wal_senders=10000"

fails with the following error:

	FATAL:  invalid value for parameter "max_wal_senders": 10000

Fortunately, there is an extra copy of this check in
InitializeMaxBackends() that we can rely on, so this commit removes
the aforementioned GUC check hooks in favor of that one.  It also
enhances the error message to clearly show the values of the
relevant GUCs and the hard-coded limit their sum may not exceed.
The downside of this change is that server startup progresses
further before failing due to such misconfigurations (thus taking
longer), but these failures are expected to be rare, so we don't
anticipate any real harm in practice.

Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/ZnMr2k-Nk5vj7T7H%40nathan
2024-07-05 14:42:55 -05:00
Tom Lane
ba8f00eef6 Improve PL/Tcl's method for choosing Tcl names of procedures.
Previously, the internal name of a PL/Tcl function was just
"__PLTcl_proc_NNNN", where NNNN is the function OID.  That's pretty
unhelpful when reading an error report.  Plus it prevents us from
testing the CONTEXT output for PL/Tcl errors, since the OIDs shown
in the regression tests wouldn't be stable.

Instead, base the internal name on the result of format_procedure(),
which will be unique in most cases.  For the edge cases where it's
not, we can append the function OID to make it unique.

Sadly, the pltcl_trigger.sql test script still has to suppress the
context reports, because they'd include trigger arguments which
contain relation OIDs per PL/Tcl's longstanding API for triggers.

I had to modify one existing test case to throw a different error
than before, because I found that Tcl 8.5 and Tcl 8.6 spell the
context message for the original error slightly differently.
We might have to make more adjustments in that vein once this
gets wider testing.

Patch by me; thanks to Pavel Stehule for the idea to use
format_procedure() rather than just the proname.

Discussion: https://postgr.es/m/890581.1717609350@sss.pgh.pa.us
2024-07-05 14:14:42 -04:00
Tom Lane
aaab3ee9c6 Doc: minor improvements for our "Brief History" chapter.
Add a link to Joe Hellerstein's paper "Looking Back at Postgres",
which is quite an interesting take on the history of Postgres.

The reference to Appendix E was written when we were still keeping
the entire release-note history there, which we stopped doing some
years ago when the O(N^2) cost of that started to become apparent.
Instead, point to the release note archives on the website.
(This per suggestion from Daniel Gustafsson.)

In passing, move the "ports12" biblioentry to be in alphabetical
order within that section.

Discussion: https://postgr.es/m/3345678.1720071633@sss.pgh.pa.us
2024-07-05 13:12:34 -04:00
Michael Paquier
4b211003ec Support loading of injection points
This can be used to load an injection point and prewarm the
backend-level cache before running it, to avoid issues if the point
cannot be loaded due to restrictions in the code path where it would be
run, like a critical section where no memory allocation can happen
(load_external_function() can do allocations when expanding a library
name).

Tests can use a macro called INJECTION_POINT_LOAD() to load an injection
point.  The test module injection_points gains some tests, and a SQL
function able to load an injection point.

Based on a request from Andrey Borodin, who has implemented a test for
multixacts requiring this facility.

Reviewed-by: Andrey Borodin
Discussion: https://postgr.es/m/ZkrBE1e2q2wGvsoN@paquier.xyz
2024-07-05 18:09:03 +09:00
Heikki Linnakangas
98347b5a3a Lift limitation that PGPROC->links must be the first field
Since commit 5764f611e1, we've been using the ilist.h functions for
handling the linked list. There's no need for 'links' to be the first
element of the struct anymore, except for one call in InitProcess
where we used a straight cast from the 'dlist_node *' to PGPROC *,
without the dlist_container() macro. That was just an oversight in
commit 5764f611e1, fix it.

There no imminent need to move 'links' from being the first field, but
let's be tidy.

Reviewed-by: Aleksander Alekseev, Andres Freund
Discussion: https://www.postgresql.org/message-id/22aa749e-cc1a-424a-b455-21325473a794@iki.fi
2024-07-05 11:21:46 +03:00
David Rowley
590b045c37 Improve memory management and performance of tuplestore.c
Here we make tuplestore.c use a generation.c memory context rather than
allocating tuples into the CurrentMemoryContext, which primarily is the
ExecutorState or PortalHoldContext memory context.  Not having a
dedicated context can cause the CurrentMemoryContext context to become
bloated when pfree'd chunks are not reused by future tuples.  Using
generation speeds up users of tuplestore.c, such as the Materialize,
WindowAgg and CTE Scan executor nodes.  The main reason for the speedup is
due to generation.c being more memory efficient than aset.c memory
contexts.  Specifically, generation does not round sizes up to the next
power of 2 value.  This both saves memory, allowing more tuples to fit in
work_mem, but also makes the memory usage more compact and fit on fewer
cachelines.  One benchmark showed up to a 22% performance increase in a
query containing a Materialize node.  Much higher gains are possible if
the memory reduction prevents tuplestore.c from spilling to disk.  This is
especially true for WindowAgg nodes where improvements of several thousand
times are possible if the memory reductions made here prevent tuplestore
from spilling to disk.

Additionally, a generation.c memory context is much better suited for this
job as it works well with FIFO palloc/pfree patterns, which is exactly how
tuplestore.c uses it.  Because of the way generation.c allocates memory,
tuples consecutively stored in tuplestores are much more likely to be
stored consecutively in memory.  This allows the CPU's hardware prefetcher
to work more efficiently as it provides a more predictable pattern to
allow cachelines for the next tuple to be loaded from RAM in advance of
them being needed by the executor.

Using a dedicated memory context for storing tuples also allows us to more
efficiently clean up the memory used by the tuplestore as we can reset or
delete the context rather than looping over all stored tuples and
pfree'ing them one by one.

Also, remove a badly placed USEMEM call in readtup_heap().  The tuple
wasn't being allocated in the Tuplestorestate's context, so no need to
adjust the memory consumed by the tuplestore there.

Author: David Rowley
Reviewed-by: Matthias van de Meent, Dmitry Dolgov
Discussion: https://postgr.es/m/CAApHDvp5Py9g4Rjq7_inL3-MCK1Co2CRt_YWFwTU2zfQix0p4A@mail.gmail.com
2024-07-05 17:51:27 +12:00
David Rowley
53abb1e0eb Fix newly introduced issue in EXPLAIN for Materialize nodes
The code added in 1eff8279d was lacking a check to see if the tuplestore
had been created.  In nodeMaterial.c this is done by ExecMaterial() rather
than by ExecInitMaterial(), so the tuplestore won't be created unless
the node has been executed at least once, as demonstrated by Alexander
in his report.

Here we skip showing any of the new EXPLAIN ANALYZE information when the
Materialize node has not been executed.

Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/fe7fc8fb-86e5-ecb0-3cb2-dd2c9a6c482f@gmail.com
2024-07-05 16:56:16 +12:00
Thomas Munro
18501841bc Add simple codepoint redirections to unaccent.rules.
Previously we searched for code points where the Unicode data file
listed an equivalent combining character sequence that added accents.
Some codepoints redirect to a single other codepoint, instead of doing
any combining.  We can follow those references recursively to get the
answer.

Per bug report #18362, which reported missing Ancient Greek characters.
Specifically, precomposed characters with oxia (from the polytonic
accent system used for old Greek) just point to precomposed characters
with tonos (from the monotonic accent system for modern Greek), and we
have to follow the extra hop to find out that they are composed with
an acute accent.

Besides those, the new rule also:

* pulls in a lot of 'Mathematical Alphanumeric Symbols', which are
  copies of the Latin and Greek alphabets and numbers rendered
  in different typefaces, and

* corrects a single mathematical letter that previously came from the
  CLDR transliteration file, but the new rule extracts from the main
  Unicode database file, where clearly the latter is right and the
  former is a wrong (reported to CLDR).

Reported-by: Cees van Zeeland <cees.van.zeeland@freedom.nl>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/18362-be6d0cfe122b6354%40postgresql.org
2024-07-05 15:25:31 +12:00
David Rowley
1eff8279d4 Add memory/disk usage for Material nodes in EXPLAIN
Up until now, there was no ability to easily determine if a Material
node caused the underlying tuplestore to spill to disk or even see how
much memory the tuplestore used if it didn't.

Here we add some new functions to tuplestore.c to query this information
and add some additional output in EXPLAIN ANALYZE to display this
information for the Material node.

There are a few other executor node types that use tuplestores, so we
could also consider adding these details to the EXPLAIN ANALYZE for
those nodes too.  Let's consider those independently from this.  Having
the tuplestore.c infrastructure in to allow that is step 1.

Author: David Rowley
Reviewed-by: Matthias van de Meent, Dmitry Dolgov
Discussion: https://postgr.es/m/CAApHDvp5Py9g4Rjq7_inL3-MCK1Co2CRt_YWFwTU2zfQix0p4A@mail.gmail.com
2024-07-05 14:05:08 +12:00
Richard Guo
aa86129e19 Support "Right Semi Join" plan shapes
Hash joins can support semijoin with the LHS input on the right, using
the existing logic for inner join, combined with the assurance that only
the first match for each inner tuple is considered, which can be
achieved by leveraging the HEAP_TUPLE_HAS_MATCH flag.  This can be very
useful in some cases since we may now have the option to hash the
smaller table instead of the larger.

Merge join could likely support "Right Semi Join" too.  However, the
benefit of swapping inputs tends to be small here, so we do not address
that in this patch.

Note that this patch also modifies a test query in join.sql to ensure it
continues testing as intended.  With this patch the original query would
result in a right-semi-join rather than semi-join, compromising its
original purpose of testing the fix for neqjoinsel's behavior for
semi-joins.

Author: Richard Guo
Reviewed-by: wenhui qiu, Alena Rybakina, Japin Li
Discussion: https://postgr.es/m/CAMbWs4_X1mN=ic+SxcyymUqFx9bB8pqSLTGJ-F=MHy4PW3eRXw@mail.gmail.com
2024-07-05 09:26:48 +09:00
Tom Lane
5a519abedd Doc: small improvements in discussion of geometric data types.
State explicitly that the coordinates in our geometric data types are
float8.  Also explain that polygons store their bounding box.

While here, fix the table of geometric data types to show type
"line"'s size correctly: it's 24 bytes not 32.  This has somehow
escaped notice since that table was made in 1998.

Per suggestion from Sebastian Skałacki.  The size error seems
important enough to justify back-patching.

Discussion: https://postgr.es/m/172000045661.706.1822177575291548794@wrigleys.postgresql.org
2024-07-04 13:23:32 -04:00
Alvaro Herrera
2ef575c780
Fix copy/paste mistake in comment
Backpatch to 17

Author: Yugo NAGATA <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/20240704134638.355ad44a445fa1e764a220cd@sranhm.sraoss.co.jp
2024-07-04 13:57:47 +02:00
Alvaro Herrera
768f0c3e21
Remove bogus assertion in pg_atomic_monotonic_advance_u64
This code wanted to ensure that the 'exchange' variable passed to
pg_atomic_compare_exchange_u64 has correct alignment, but apparently
platforms don't actually require anything that doesn't come naturally.

While messing with pg_atomic_monotonic_advance_u64: instead of using
Max() to determine the value to return, just use
pg_atomic_compare_exchange_u64()'s return value to decide; also, use
pg_atomic_compare_exchange_u64 instead of the _impl version; also remove
the unnecessary underscore at the end of variable name "target".

Backpatch to 17, where this code was introduced by commit bf3ff7bf83bc.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/36796438-a718-cf9b-2071-b2c1b947c1b5@gmail.com
2024-07-04 13:25:31 +02:00
Daniel Gustafsson
ab0ae64320 doc: Specify when ssl_prefer_server_ciphers was added
The ssl_prefer_server_ciphers setting is quite important from a
security point of view, so simply stating that older versions
doesn't have it isn't very helpful.  This adds the version when
the GUC was added to help readers.

Backpatch to all supported versions since this setting has been
around since 9.4.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/5D7E0F5E-E620-4D54-8788-66D421AC76F0@yesql.se
Backpatch-through: v12
2024-07-04 11:38:37 +02:00
Michael Paquier
4564f1cebd Add pg_get_acl() to get the ACL for a database object
This function returns the ACL for a database object, specified by
catalog OID and object OID.  This is useful to be able to
retrieve the ACL associated to an object specified with a
(class_id,objid) couple, similarly to the other functions for object
identification, when joined with pg_depend or pg_shdepend.

Original idea by Álvaro Herrera.

Bump catalog version.

Author: Joel Jacobson
Reviewed-by: Isaac Morland, Michael Paquier, Ranier Vilela
Discussion: https://postgr.es/m/80b16434-b9b1-4c3d-8f28-569f21c2c102@app.fastmail.com
2024-07-04 17:09:06 +09:00
Amit Langote
3a8a1f3254 SQL/JSON: Fix some obsolete comments.
JSON_OBJECT(), JSON_OBJETAGG(), JSON_ARRAY(), and JSON_ARRAYAGG()
added in 7081ac46ace are not transformed into direct calls to
user-defined functions as the comments claim. Fix by mentioning
instead that they are transformed into JsonConstructorExpr nodes,
which may call them, for example, for the *AGG() functions.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/058c856a-e090-ac42-ff00-ffe394f52a87%40gmail.com
Backpatch-through: 16
2024-07-04 16:05:35 +09:00
Michael Paquier
b81a71aa05 Assign error codes where missing for user-facing failures
All the errors triggered in the code paths patched here would cause the
backend to issue an internal_error errcode, which is a state that should
be used only for "can't happen" situations.  However, these code paths
are reachable by the regression tests, and could be seen by users in
valid cases.  Some regression tests expect internal errcodes as they
manipulate the backend state to cause corruption (like checksums), or
use elog() because it is more convenient (like injection points), these
have no need to change.

This reduces the number of internal failures triggered in a check-world
by more than half, while providing correct errcodes for these valid
cases.

Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/Zic_GNgos5sMxKoa@paquier.xyz
2024-07-04 09:48:40 +09:00
Alexander Korotkov
6897f0ec02 Optimize memory access in GetRunningTransactionData()
e85662df44 made GetRunningTransactionData() calculate the oldest running
transaction id within the current database.  This commit optimized this
calculation by performing a cheap transaction id comparison before fetching
the process database id, while the latter could cause extra cache misses.

Reported-by: Noah Misch
Discussion: https://postgr.es/m/20240630231816.bf.nmisch%40google.com
2024-07-04 02:05:37 +03:00
Alexander Korotkov
6c1af5482e Fix typo in GetRunningTransactionData()
e85662df44 made GetRunningTransactionData() calculate the oldest running
transaction id within the current database.  However, because of the typo,
the new code uses oldestRunningXid instead of oldestDatabaseRunningXid
in comparison before updating oldestDatabaseRunningXid.  This commit fixes
that issue.

Reported-by: Noah Misch
Discussion: https://postgr.es/m/20240630231816.bf.nmisch%40google.com
Backpatch-through: 17
2024-07-04 02:05:27 +03:00
David Rowley
4331a11c62 Remove incorrect Asserts in buffile.c
Both BufFileSize() and BufFileAppend() contained Asserts to ensure the
given BufFile(s) had a valid fileset.  A valid fileset isn't required in
either of these functions, so remove the Asserts and adjust the
comments accordingly.

This was noticed while work was being done on a new patch to call
BufFileSize() on a BufFile without a valid fileset.  It seems there's
currently no code in the tree which could trigger these Asserts, so no
need to backpatch this, for now.

Reviewed-by: Peter Geoghegan, Matthias van de Meent, Tom Lane
Discussion: https://postgr.es/m/CAApHDvofgZT0VzydhyGH5MMb-XZzNDqqAbzf1eBZV5HDm3%2BosQ%40mail.gmail.com
2024-07-04 09:44:34 +12:00
Nathan Bossart
2329cad1b9 Improve performance of binary_upgrade_set_pg_class_oids().
This function generates the commands that preserve the OIDs and
relfilenodes of relations during pg_upgrade.  It is called once per
relevant relation, and each such call executes a relatively
expensive query to retrieve information for a single pg_class_oid.
This can cause pg_dump to take significantly longer when
--binary-upgrade is specified, especially when there are many
tables.

This commit improves the performance of this function by gathering
all the required pg_class information with a single query at the
beginning of pg_dump.  This information is stored in a sorted array
that binary_upgrade_set_pg_class_oids() can bsearch() for what it
needs.  This follows a similar approach as commit d5e8930f50, which
introduced a sorted array for role information.

With this patch, 'pg_dump --binary-upgrade' will use more memory,
but that isn't expected to be too egregious.  Per the mailing list
discussion, folks feel that this is worth the trade-off.

Reviewed-by: Corey Huinker, Michael Paquier, Daniel Gustafsson
Discussion: https://postgr.es/m/20240418041712.GA3441570%40nathanxps13
2024-07-03 14:21:50 -05:00
Nathan Bossart
6e1c4a03a9 Remove is_index parameter from binary_upgrade_set_pg_class_oids().
Since commit 9a974cbcba, this function retrieves the relkind before
it needs to know whether the relation is an index, so we no longer
need callers to provide this information.

Suggested-by: Daniel Gustafsson
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/20240418041712.GA3441570%40nathanxps13
2024-07-03 10:58:26 -05:00
Heikki Linnakangas
f3412a61f3 Avoid 0-length memcpy to NULL with EXEC_BACKEND
memcpy(NULL, src, 0) is forbidden by POSIX, even though every
production version of libc allows it. Let's be tidy.

Per report from Thomas Munro, running UBSan with EXEC_BACKEND.
Backpatch to v17, where this code was added.

Discussion: https://www.postgresql.org/message-id/CA%2BhUKG%2Be-dV7YWBzfBZXsgovgRuX5VmvmOT%2Bv0aXiZJ-EKbXcw@mail.gmail.com
2024-07-03 15:58:14 +03:00
Heikki Linnakangas
a06e8f84a1 Tighten check for --forkchild argument when spawning child process
Commit aafc05de1b removed all the other --fork* arguments. Altough
this is inconsequential, backpatch to v17 since this is new.

Author: Nathan Bossart
Discussion: https://www.postgresql.org/message-id/ZnCCEN0l3qWv-XpW@nathan
2024-07-03 15:53:30 +03:00
Amit Kapila
ae395f0f7e Fix the testcase introduced in commit 81d20fbf7a.
The failed test was syncing failover replication slot to standby to test
that we remove such slots after the standby is converted to subscriber by
pg_createsubscriber.

In one of the buildfarm members, the sync of the slot failed because the
LSN on the standby was before the syncslot's LSN. We need to wait for
standby to catch up before trying to sync the slot with
pg_sync_replication_slots().

The other buildfarm failed because autovacuum generated a xid which is
replicated to the standby at some random point making slots at primary
lag behind standby during slot sync.

Both these failures wouldn't have occurred if we had used built-in
slotsync worker as it would have waited for the standby to sync with
primary but for this test, it is sufficient to use
pg_sync_replication_slots().

Reported-by: Alexander Lakhin as per buildfarm
Author: Kuroda Hayato
Reviewed-by: Amit Kapila
Backpatch-through: 17
Discussion: https://postgr.es/m/0dffca12-bf17-4a7a-334d-225569de5e6e@gmail.com
Discussion: https://postgr.es/m/OSBPR01MB25528300C71FDD83EA1DCA12F5DD2@OSBPR01MB2552.jpnprd01.prod.outlook.com
2024-07-03 15:04:59 +05:30
Michael Paquier
9fd0252579 Replace hardcoded identifiers of pgstats file by #defines
This changes pgstat.c so as the three types of entries that can exist in
a pgstats file are not hardcoded anymore, replacing them with
descriptively-named macros, when reading and writing stats files:
- 'N' for named entries, like replication slot stats.
- 'S' for entries identified by a hash.
- 'E' for the end-of-file

This has come up while working on making this area of the code more
pluggable.  The format of the stats file is unchanged, hence there is no
need to bump PGSTAT_FILE_FORMAT_ID.

Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Zmqm9j5EO0I4W8dx@paquier.xyz
2024-07-03 13:09:20 +09:00
Michael Paquier
dd569214aa Clean up more unused variables in perl code
This is a continuation of 0c1aca461481, with some cleanup in:
- msvc_gendef.pl
- pgindent
- 005_negotiate_encryption.pl, as of an oversight of d39a49c1e459 that
has removed %params in test_matrix(), making also $server_config
useless.

Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87wmm4dkci.fsf@wibble.ilmari.org
2024-07-03 12:43:57 +09:00
Nathan Bossart
dec9d4acdb Add CODE_OF_CONDUCT.md, CONTRIBUTING.md, and SECURITY.md.
These "community health files" provide important information about
the project and will be displayed prominently on the PostgreSQL
GitHub mirror.  For now, they just point to the website, but we may
want to expand on the content in the future.

Reviewed-by: Peter Eisentraut, Alvaro Herrera, Tom Lane
Discussion: https://postgr.es/m/20240417023609.GA3228660%40nathanxps13
2024-07-02 13:03:58 -05:00
Heikki Linnakangas
eb21f5bc67 Remove redundant SetProcessingMode(InitProcessing) calls
After several refactoring iterations, auxiliary processes are no
longer initialized from the bootstrapper. Using the InitProcessing
mode for initializing auxiliary processes is more appropriate. Since
the global variable Mode is initialized to InitProcessing, we can just
remove the redundant calls of SetProcessingMode(InitProcessing).

Author: Xing Guo <higuoxing@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACpMh%2BDBHVT4xPGimzvex%3DwMdMLQEu9PYhT%2BkwwD2x2nu9dU_Q%40mail.gmail.com
2024-07-02 20:14:40 +03:00
Heikki Linnakangas
4d22173ec0 Move bgworker specific logic to bgworker.c
For clarity, we've been slowly moving functions that are not called
from the postmaster process out of postmaster.c.

Author: Xing Guo <higuoxing@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACpMh%2BDBHVT4xPGimzvex%3DwMdMLQEu9PYhT%2BkwwD2x2nu9dU_Q%40mail.gmail.com
2024-07-02 20:12:05 +03:00
Nathan Bossart
8213df9eff pg_dump: Remove some unused return values.
getSchemaData() does not use the return values of many of its get*
helper functions because they store the data elsewhere.  For
example, commit 92316a4582 introduced a separate hash table for
dumpable objects that said helper functions populate.  This commit
changes these functions to return void and removes their "int *"
parameters that returned the number of objects found.

Reviewed-by: Neil Conway, Tom Lane, Daniel Gustafsson
Discussion: https://postgr.es/m/ZmCAtVaOrHpf31PJ%40nathan
2024-07-02 11:22:06 -05:00
Daniel Gustafsson
e930c872b6 Use safe string copy routine
Using memcpy with strlen as the size parameter will not take the
NULL terminator into account, relying instead on the destination
buffer being properly initialized. Replace with strlcpy which is
a safer alternative, and more in line with how we handle copying
strings elsewhere.

Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQApAsbLsQ+gGiw-hT+JwGhgogFa_=5NUkgFO6kOPxyNidQ@mail.gmail.com
2024-07-02 11:16:56 +02:00
Peter Eisentraut
da3ea048ca Remove too demanding test
Remove the test from commit 9c2e660b07.  This test ends up allocating
quite a bit of memory, which can make the test fail with out of memory
errors on some build farm machines.
2024-07-02 10:43:12 +02:00