mirrors/postgres
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add WAL mmap() mention.

Browse Source
This commit is contained in:
Bruce Momjian 2001-10-12 17:35:10 +00:00
parent 68966634b2
commit c0b2bbf42e
1 changed files with 103 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

103
doc/TODO.detail/mmap
View File

@ -379,3 +379,106 @@ TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
From pgsql-hackers-owner+M13750=candle.pha.pa.us=pgman@postgresql.org Mon Oct 1 05:59:15 2001
Return-path: <pgsql-hackers-owner+M13750=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f919xF512590
for <pgman@candle.pha.pa.us>; Mon, 1 Oct 2001 05:59:15 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f919xA207817
for <pgman@candle.pha.pa.us>; Mon, 1 Oct 2001 04:59:10 -0500 (CDT)
(envelope-from pgsql-hackers-owner+M13750=candle.pha.pa.us=pgman@postgresql.org)
Received: from mrsgntmail01.mediaring.com.sg (mserver.mediaring.com.sg [203.208.141.175])
by postgresql.org (8.11.3/8.11.4) with ESMTP id f919rE320926
for <pgsql-hackers@postgreSQL.org>; Mon, 1 Oct 2001 05:53:15 -0400 (EDT)
(envelope-from jana-reddy@mediaring.com.sg)
Received: by MRSGNTMAIL01 with Internet Mail Service (5.5.2650.21)
id <PMTCM7SJ>; Mon, 1 Oct 2001 18:03:34 +0800
Received: from mediaring.com.sg (10.1.0.131 [10.1.0.131]) by mrsgntmail01.mediaring.com.sg with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id PMTCM7SH; Mon, 1 Oct 2001 18:03:25 +0800
From: Janardhana Reddy <jana-reddy@mediaring.com.sg>
To: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>,
janareddy
<jana-reddy@mediaring.com.sg>
Message-ID: <3BB83DF0.8946973@mediaring.com.sg>
Date: Mon, 01 Oct 2001 17:57:04 +0800
X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.0 i686)
X-Accept-Language: en
MIME-Version: 1.0
Subject: Re: [HACKERS] PERFORMANCE IMPROVEMENT by mapping WAL FILES
References: <200109282137.f8SLbpm01890@candle.pha.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: ORr
I have just completed the functional testing the WAL using mmap , it is
working fine, I have tested by commenting out the "CreateCheckPoint "
functionality so that
when i kill the postgres and restart it will redo all the records from the
WAL log file which
is updated using mmap.
Just i need to clean code and to do some stress testing.
By the end of this week i should able to complete the stress test and
generate the patch file .
As Tom Lane mentioned i see the problem in portability to all platforms,
what i propose is to use mmap for only WAL for some platforms like
linux,freebsd etc . For other platforms we can use the existing method by
slightly modifying the
write() routine to write only the modified part of the page.
Regards
jana
>
>
> OK, I have talked to Tom Lane about this on the phone and we have a few
> ideas.
>
> Historically, we have avoided mmap() because of portability problems,
> and because using mmap() to write to large tables could consume lots of
> address space with little benefit. However, I perhaps can see WAL as
> being a good use of mmap.
>
> First, there is the issue of using mmap(). For OS's that have the
> mmap() MAP_SHARED flag, different backends could mmap the same file and
> each see the changes. However, keep in mind we still have to fsync()
> WAL, so we need to use msync().
>
> So, looking at the benefits of using mmap(), we have overhead of
> different backends having to mmap something that now sits quite easily
> in shared memory. Now, I can see mmap reducing the copy from user to
> kernel, but there are other ways to fix that. We could modify the
> write() routines to write() 8k on first WAL page write and later write
> only the modified part of the page to the kernel buffers. The old
> kernel buffer is probably still around so it is unlikely to require a
> read from the file system to read in the rest of the page. This reduces
> the write from 8k to something probably less than 4k which is better
> than we can do with mmap.
>
> I will add a TODO item to this effect.
>
> As far as reducing the write to disk from 8k to 4k, if we have to
> fsync/msync, we have to wait for the disk to spin to the proper location
> and at that point writing 4k or 8k doesn't seem like much of a win.
>
> In summary, I think it would be nice to reduce the 8k transfer from user
> to kernel on secondary page writes to only the modified part of the
> page. I am uncertain if mmap() or anything else will help the physical
> write to the disk.
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman@candle.pha.pa.us | (610) 853-3000
> + If your life is a hard drive, | 830 Blythe Avenue
> + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?
http://archives.postgresql.org