mirrors/postgres
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b538215d5d
postgres/contrib/fuzzystrmatch/README.fuzzystrmatch

145 lines
4.0 KiB
Plaintext
Raw Normal View History

Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
/*
* fuzzystrmatch.c
*
* Functions for "fuzzy" comparison of strings
*
Jim C. Nasby wrote: > Second argument to metaphone is suposed to set the limit on the > number of characters to return, but it breaks on some phrases: > > usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from > (select 'Hello world'::varchar AS a) a; > HLW | HLWR | HLWRLT > > usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from > (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; > AKM | AKMKS | AKMKSMMRL > > In every case I've found that does this, the 4th and 5th letters are > always 'KS'. Nice catch. There was a bug in the original metaphone algorithm from CPAN. Patch attached (while I was at it I updated my email address, changed the copyright to PGDG, and removed an unnecessary palloc). Here's how it looks now: regression=# select metaphone(a,4) from (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; metaphone ----------- AKMK (1 row) regression=# select metaphone(a,5) from (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; metaphone ----------- AKMKS (1 row) Joe Conway
2003-06-25 02:59:46 +04:00
* Joe Conway <mail@joeconway.com>
*
Update copyright for 2006. Update scripts.
2006-03-05 18:59:11 +03:00
* Copyright (c) 2001-2006, PostgreSQL Global Development Group
Jim C. Nasby wrote: > Second argument to metaphone is suposed to set the limit on the > number of characters to return, but it breaks on some phrases: > > usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from > (select 'Hello world'::varchar AS a) a; > HLW | HLWR | HLWRLT > > usps=# select metaphone(a,3),metaphone(a,4),metaphone(a,20) from > (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; > AKM | AKMKS | AKMKSMMRL > > In every case I've found that does this, the 4th and 5th letters are > always 'KS'. Nice catch. There was a bug in the original metaphone algorithm from CPAN. Patch attached (while I was at it I updated my email address, changed the copyright to PGDG, and removed an unnecessary palloc). Here's how it looks now: regression=# select metaphone(a,4) from (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; metaphone ----------- AKMK (1 row) regression=# select metaphone(a,5) from (select 'A A COMEAUX MEMORIAL'::varchar AS a) a; metaphone ----------- AKMKS (1 row) Joe Conway
2003-06-25 02:59:46 +04:00
* ALL RIGHTS RESERVED;
Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
*
* levenshtein()
* -------------
* Written based on a description of the algorithm by Michael Gilleland
* found at http://www.merriampark.com/ld.htm
* Also looked at levenshtein.c in the PHP 4.0.6 distribution for
* inspiration.
*
* metaphone()
* -----------
* Modified for PostgreSQL by Joe Conway.
* Based on CPAN's "Text-Metaphone-1.96" by Michael G Schwern <schwern@pobox.com>
* Code slightly modified for use as PostgreSQL function (palloc, elog, etc).
* Metaphone was originally created by Lawrence Philips and presented in article
* in "Computer Language" December 1990 issue.
*
Add double metaphone code from Andrew Dunstan. Also change metaphone so that an empty input string causes an empty output string to be returned, instead of throwing an ERROR -- per complaint from Aaron Hillegass, and consistent with double metaphone. Fix examples in README.soundex pointed out by James Robinson.
2004-07-01 07:25:48 +04:00
* dmetaphone() and dmetaphone_alt()
* ---------------------------------
* A port of the DoubleMetaphone perl module by Andrew Dunstan. See dmetaphone.c
* for more detail.
*
Sorry - I should have gotten to this sooner. Here's a patch which you should be able to apply against what you just committed. It rolls soundex into fuzzystrmatch. Remove soundex/metaphone and merge into fuzzystrmatch. Joe Conway
2001-08-07 22:16:01 +04:00
* soundex()
* -----------
* Folded existing soundex contrib into this one. Renamed text_soundex() (C function)
* to soundex() for consistency.
*
The attached patch implements the soundex difference function which compares two strings' soundex values for similarity, from Kris Jurka. Also mark the text_soundex() function as STRICT, to avoid crashing on NULL input.
2005-01-26 11:04:04 +03:00
* difference()
* ------------
* Return the difference between two strings' soundex values. Kris Jurka
*
Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
* Permission to use, copy, modify, and distribute this software and its
* documentation for any purpose, without fee, and without a written agreement
* is hereby granted, provided that the above copyright notice and this
* paragraph and the following two paragraphs appear in all copies.
*
* IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR
* DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
* LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
* DOCUMENTATION, EVEN IF THE AUTHOR OR DISTRIBUTORS HAVE BEEN ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES,
* INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
* AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
* ON AN "AS IS" BASIS, AND THE AUTHOR AND DISTRIBUTORS HAS NO OBLIGATIONS TO
* PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
*
*/
Add double metaphone code from Andrew Dunstan. Also change metaphone so that an empty input string causes an empty output string to be returned, instead of throwing an ERROR -- per complaint from Aaron Hillegass, and consistent with double metaphone. Fix examples in README.soundex pointed out by James Robinson.
2004-07-01 07:25:48 +04:00
Version 0.3 (30 June, 2004):
Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
Release Notes:
Add double metaphone code from Andrew Dunstan. Also change metaphone so that an empty input string causes an empty output string to be returned, instead of throwing an ERROR -- per complaint from Aaron Hillegass, and consistent with double metaphone. Fix examples in README.soundex pointed out by James Robinson.
2004-07-01 07:25:48 +04:00
Version 0.3
- added double metaphone code from Andrew Dunstan
- change metaphone so that an empty input string causes an empty
output string to be returned, instead of throwing an ERROR
- fixed examples in README.soundex
Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
Sorry - I should have gotten to this sooner. Here's a patch which you should be able to apply against what you just committed. It rolls soundex into fuzzystrmatch. Remove soundex/metaphone and merge into fuzzystrmatch. Joe Conway
2001-08-07 22:16:01 +04:00
Version 0.2
- folded soundex contrib into this one
Per this discussion, here's a patch to implement both levenshtein() and metaphone() in a contrib. There seem to be a fair number of different approaches to both of these algorithms. I used the simplest case for levenshtein which has a cost of 1 for any character insertion, deletion, or substitution. For metaphone, I adapted the same code from CPAN that the PHP folks did. A couple of questions: 1. Does it make sense to fold the soundex contrib together with this one? 2. I was debating trying to add multibyte support to levenshtein (it would make no sense at all for metaphone), but a quick search through the contrib directory found no hits on the word MULTIBYTE. Should worry about adding multibyte support to levenshtein()? Joe Conway
2001-08-07 20:47:43 +04:00
Version 0.1
- initial release
Installation:
Place these files in a directory called 'fuzzystrmatch' under 'contrib' in the PostgreSQL source tree. Then run:
make
make install
You can use fuzzystrmatch.sql to create the functions in your database of choice, e.g.
psql -U postgres template1 < fuzzystrmatch.sql
installs following functions into database template1:
levenshtein() - calculates the levenshtein distance between two strings
metaphone() - calculates the metaphone code of an input string
Documentation
==================================================================
Name
levenshtein -- calculates the levenshtein distance between two strings
Synopsis
levenshtein(text source, text target)
Inputs
source
any text string, 255 characters max, NOT NULL
target
any text string, 255 characters max, NOT NULL
Outputs
Returns int
Example usage
select levenshtein('GUMBO','GAMBOL');
==================================================================
Name
metaphone -- calculates the metaphone code of an input string
Synopsis
metaphone(text source, int max_output_length)
Inputs
source
any text string, 255 characters max, NOT NULL
max_output_length
maximum length of the output metaphone code; if longer, the output
is truncated to this length
Outputs
Returns text
Example usage
select metaphone('GUMBO',4);
==================================================================
-- Joe Conway
Reference in New Issue Copy Permalink