An attached patch corrects problem of this bug and fractional second.
The handling of time zone was as follows:
(a) with time zone
using SimpleDateFormat("yyyy-MM-dd HH:mm:ss z")
(b) without time zone
using SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
About problem of fractional second,
Fractional second was changed from milli-second to nano-second
a get on a bytea value the code was running the raw value from the server
through character set conversion, which if the character set was SQL_ASCII
would cause all 8bit characters to become ?'s.
wasn't updated to handle more than two decimal digits for fractional seconds
that now are possible in 7.2. This patch fixes the timestamp parsing logic.
I have built and tested on both jdbc1 and jdbc2.
The bug was that any insert or update would fail if the returned oid was
larger than a signed int. Since OIDs are unsigned int's it was
a bug that the code used a java signed int to deal with the values. The bug
would result in the error message: "Unable to fathom update count".
While fixing the bug, it became apparent that other code made a similar
assumption about OIDs being signed ints. Therefore some methods that returned
or took OIDs are arguements also needed to be changed.
Since we are so close to the 7.2 release I have added new methods that
return longs and deprecated the old methods returning ints. Therefore all
old code should still work without requiring a code change to cast from long to int. Also note that the methods below are PostgreSQL specific extensions to
the JDBC api are are not part of the spec from Sun, thus it is unlikely that
they are used much or at all.
The deprecated methods are:
ResultSet.getInsertedOID()
Statement.getInsertedOID()
Serialize.store()
Connection.putObject()
and are replaced by:
ResultSet.getLastOID()
Statement.getLastOID()
Serialize.storeObject()
Connection.storeObject()
All the deprecated methods returned int, while their replacements return long
This patch also fixes two comments in MD5Digest that the author Jeremy Wohl
submitted.
--Barry
There's also a little fix for the getRow() method. While fixing
absolute(), I noticed that getRow() wasn't quite following the spec: it
wasn't returning 0 when the ResultSet wasn't positioned on a row. I've
started a ResultSet test case and included it as well.
Liam Stewart
That patch broke the ability to read data from binary cursors.
--Barry Lind
Modified Files:
pgsql/src/interfaces/jdbc/org/postgresql/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc1/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc1/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/UpdateableResultSet.java
select 'id' as xxx from table
The issue is:
When the driver gets a data type which does not map into the SQL.Types
it attempts to load the object into a java object. Eventually throwing
an exception indicating that the type "unknown" was not found.
Since the backend defaults "unknown" types to text it was suggested that
the jdbc driver do the same.
This patch does just that.
I have tested it on the above select statement as well as a small
program that serializes, and deserializes a class
Dave Cramer
> null bytes to be literally '\0', the following can happen:
> 1. User inputs string value as "<null byte>##" where ## are digits in the
> range of 0 to 7.
> 2. PQescapeString converts this to "\0##"
> 3. Escaped string is used in a context that causes "\0##" to be evaluated as
> an octal escape sequence.
I agree that this is a problem, though it is not possible to do
anything harmful with it. In addition, it only occurs if there are
any NUL characters in its input, which is very unlikely if you are
using C strings.
The patch below addresses the issue by removing escaping of \0
characters entirely.
> If the goal is to "safely" encode null bytes, and preserve the rest of the
> string as it was entered, I think the null bytes should be escaped as \\000
> (note that if you simply use \000 the same string truncation problem
> occurs).
We can't do that, this would require 4n + 1 bytes of storage for the
result, breaking the interface.
Florian Weimer
This patch does the following:
- Adds binary datatype support (bytea)
- Changes getXXXStream()/setXXXStream() methods to be spec compliant
- Adds ability to revert to old behavior
Details:
Adds support for the binary type bytea. The ResultSet.getBytes() and
PreparedStatement.setBytes() methods now work against columns of bytea
type. This is a change in behavior from the previous code which assumed
the column type was OID and thus a LargeObject. The new behavior is
more complient with the JDBC spec as BLOB/CLOB are to be used for
LargeObjects and the getBytes()/setBytes() methods are for the databases
binary datatype (which is bytea in postgres).
Changes the behavior of the getBinaryStream(), getAsciiStream(),
getCharacterStream(), getUnicodeStream() and their setXXXStream()
counterparts. These methos now work against either the bytea type
(BinaryStream) or the text types (AsciiStream, CharacterStream,
UnicodeStream). The previous behavior was that these all assumed the
underlying column was of type OID and thus a LargeObject. The
spec/javadoc for these methods indicate that they are for LONGVARCHAR
and LONGVARBINARY datatypes, which are distinct from the BLOB/CLOB
datatypes. Given that the bytea and text types support upto 1G, they
are the LONGVARBINARY and LONGVARCHAR datatypes in postgres.
Added support for turning off the above new functionality. Given that
the changes above are not backwardly compatible (however they are more
spec complient), I added the ability to revert back to the old behavior.
The Connection now takes an optional parameter named 'compatible'. If
the value of '7.1' is passed, the driver reverts to the 7.1 behavior.
If the parameter is not passed or the value '7.2' is passed the behavior
is the new behavior. The mechanism put in place can be used in the
future when/if similar needs arise to change behavior. This is
patterned after how Oracle does this (i.e. Oracle has a 'compatible'
parameter that behaves in a similar manner).
Misc fixes. Cleaned up a few things I encountered along the way.
Note that in testing the patch I needed to ignore whitespace differences
in order to get it to apply cleanly (i.e. patch -l -i byteapatch.diff).
Also this patch introduces a new file
(src/interfaces/jdbc/org/postgresql/util/PGbytea.java).
Barry Lind
This patch moves the logic that looks up TypeOid, PGTypeName, and
SQLTypeName from Field to Connection. It is moved to connection since
it needs to differ from the jdbc1 to jdbc2 versions and Connection
already has different subclasses for the two driver versions. It also
made sense to move the logic to Connection as some of the logic was
already there anyway.
Barry Lind
following email.
> > The problem: When I call getBigDecimal() on a ResultSet, it
> > sometimes throws an exception:
> >
> > Bad BigDecimal 174.50
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:373)
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:984)
> > ...blah blah blah...
> > org.postgresql.util.PSQLException: Bad BigDecimal 174.50
Barry Lind
attempt at a patch to 7.1.2 to support Array.
[I think I've solved the mangled patch problem. Hotmail seems to
try to format the text file, so gzipping it should solve this
problem.]
In this patch I've incorporated Barry's feedback. Specifically:
1) OIDs are no longer hard-coded into Array.java. In order to
support this change I added a getOID(String) method to Field.java
which receives a PostgreSQL field type and returns a value from
java.sql.Types. I couldn't get away from using OIDs altogether
because the JDBC spec for Array specifies that some methods return
a ResultSet. This requires I construct Field objects,
which means I need OIDs. At least this approach doesn't hard
code these values. A Hashtable cache has been added to Field
so that an SQL lookup isn't necessary (following the model already
in Field.java).
2) Rewired the base formatting code in ResultSet.java to use 'to'
methods, which are then exposed as static methods in ResultSet.
These methods are used in Array to format the data without
duplications in the code.
3) Artifact call to first() in ResultSet.getArray() removed.
Greg Zoller
Here's a patch against the current CVS. The changes from the previous
patch are mostly related to the changed interface for PG_Stream.
Anders Bengtsson
submit. These were done for the jdbc2 driver. The first one is for support
of the Types.BIT in the PreparedStatement class. The following lines need to be
inserted in the switch statment, at around line 530:
(Prepared statment, line 554, before the default: switch
case Types.BIT:
if (x instanceof Boolean) {
set(parameterIndex, ((Boolean)x).booleanValue() ? "TRUE" : "FALSE");
} else {
throw new PSQLException("postgresql.prep.type");
}
break;
The second one is dealing with blobs,
inserted in PreparedStatemant.java (After previous patch line, 558):
case Types.BINARY:
case Types.VARBINARY:
setObject(parameterIndex,x);
break;
and in ResultSet.java (Around line 857):
case Types.BINARY:
case Types.VARBINARY:
return getBytes(columnIndex);
Ned Wolpert <ned.wolpert@knowledgenet.com>
database, and often need the latest timestamp, but want to
format it as a date. With 7.0.x, I just
select ts from foo order by ts desc limit 1
and in java: d = res.getDate(1);
but this fails everywhere in my code now :(
http://java.sun.com/j2se/1.3/docs/guide/jdbc/spec/jdbc-spec.frame7.html
says
The ResultSet.getXXX methods will attempt to
convert whatever SQL type was returned by the
database to whatever Java type is returned by
the getXXX method.
Palle Girgensohn
(1.22) of interfaces/jdbc/org/postgresql/jdbc2/ResultSet.java. That
change removed a line that set the variable s to the value of the
stringbuffer. This fix changes the following if checks to check the
length of the stringbuffer instead of s, since s no longer contains the
string the if conditions are expecting.
The bug manifests itself in getTimestamp() loosing the timezone
information of timestamps selected from the database, thereby causing
the time to be incorrect.
Barry Lind
Fri Feb 17 15:11:00 GMT 2001 peter@retep.org.uk
- Reduced the object overhead in PreparedStatement by reusing the same
StringBuffer object throughout. Similarly SimpleDateStamp's are alse
reused in a thread save manner.
- Implemented in PreparedStatement: setNull(), setDate/Time/Timestamp
using Calendar, setBlob(), setCharacterStream()
- Clob's are now implemented in ResultSet & PreparedStatement!
- Implemented a lot of DatabaseMetaData & ResultSetMetaData methods.
We have about 18 unimplemented methods left in JDBC2 at the current
time.
Tue Feb 13 16:33:00 GMT 2001 peter@retep.org.uk
- More TestCases implemented. Refined the test suite api's.
- Removed need for SimpleDateFormat in ResultSet.getDate() improving
performance.
- Rewrote ResultSet.getTime() so that it uses JDK api's better.
Tue Feb 13 10:25:00 GMT 2001 peter@retep.org.uk
- Added MiscTest to hold reported problems from users.
- Fixed PGMoney.
- JBuilder4/JDBCExplorer now works with Money fields. Patched Field &
ResultSet (lots of methods) for this one. Also changed cash/money to
return type DOUBLE not DECIMAL. This broke JBuilder as zero scale
BigDecimal's can't have decimal places!
- When a Statement is reused, the previous ResultSet is now closed.
- Removed deprecated call in ResultSet.getTime()
Thu Feb 08 18:53:00 GMT 2001 peter@retep.org.uk
- Changed a couple of settings in DatabaseMetaData where 7.1 now
supports those features
- Implemented the DatabaseMetaData TestCase.
Wed Feb 07 18:06:00 GMT 2001 peter@retep.org.uk
- Added comment to Connection.isClosed() explaining why we deviate from
the JDBC2 specification.
- Fixed bug where the Isolation Level is lost while in autocommit mode.
- Fixed bug where several calls to getTransactionIsolationLevel()
returned the first call's result.
- Fixed bug where Statement.setMaxRows() was a global setting. Now
limited to just itself.
- Changed LargeObject.read(byte[],int,int) to return the actual number
of bytes read (used to be void).
- LargeObject now supports InputStream's!
- PreparedStatement.setBinaryStream() now works!
- ResultSet.getBinaryStream() now returns an InputStream that doesn't
copy the blob into memory first!
- Connection.isClosed() now tests to see if the connection is still alive
rather than if it thinks it's alive.
objects that Thomas pointed out might be a problem.
PPS. I have included and updated the comments from the original patch
request to reflect the changes made in this revised patch.
> Attached is a set of patches for a couple of bugs dealing with
> timestamps in JDBC.
>
> Bug#1) Incorrect timestamp stored in DB if client timezone different
> than DB.
> The buggy implementation of setTimestamp() in PreparedStatement simply
> used the toString() method of the java.sql.Timestamp object to convert
> to a string to send to the database. The format of this is yyyy-MM-dd
> hh:mm:ss.SSS which doesn't include any timezone information. Therefore
> the DB assumes its timezone since none is specified. That is OK if the
> timezone of the client and server are the same, however if they are
> different the wrong timestamp is received by the server. For example if
> the client is running in timezone GMT and wants to send the timestamp
> for noon to a server running in PST (GMT-8 hours), then the server will
> receive 2000-01-12 12:00:00.0 and interprete it as 2000-01-12
> 12:00:00-08 which is 2000-01-12 04:00:00 in GMT. The fix is to send a
> format to the server that includes the timezone offset. For simplicity
> sake the fix uses a SimpleDateFormat object with its timezone set to GMT
> so that '+00' can be used as the timezone for postgresql. This is done
> as SimpleDateFormat doesn't support formating timezones in the way
> postgresql expects.
>
> Bug#2) Incorrect handling of partial seconds in getting timestamps from
> the DB
>
> When the SimpleDateFormat object parses a string with a format like
> yyyy-MM-dd hh:mm:ss.SS it expects the fractional seconds to be three
> decimal places (time precision in java is miliseconds = three decimal
> places). This seems like a bug in java to me, but it is unlikely to be
> fixed anytime soon, so the postgresql code needed modification to
> support the java behaviour. So for example a string of '2000-01-12
> 12:00:00.12-08' coming from the database was being converted to a
> timestamp object with a value of 2000-01-12 12:00:00.012GMT-08:00. The
> fix was to check for a '.' in the string and if one is found append on
> an extra zero to the fractional seconds part.
>
>
> I also did some cleanup in ResultSet.getTimestamp(). This method has
> had multiple patches applied some of which resulted in code that was no
> longer needed. For example the ISO timestamp format that postgresql
> uses specifies the timezone as an offset like '-08'. Code was added at
> one point to convert the postgresql format to the java one which is
> GMT-08:00, however the old code was left around which did nothing. So
> there was code that looked for yyyy-MM-dd hh:mm:sszzzzzzzzz and
> yyyy-MM-dd hh:mm:sszzz. This second format would never be encountered
> because zzz (i.e. -08) would be converted into the former (also note
> that the SimpleDateFormat object treats zzzzzzzzz and zzz the same, the
> number of z's does not matter).
>
>
> There was another problem/fix mentioned on the email lists today by
> mcannon@internet.com which is also fixed by this patch:
>
> Bug#3) Fractional seconds lost when getting timestamp from the DB
> A patch by Jan Thomea handled the case of yyyy-MM-dd hh:mm:sszzzzzzzzz
> but not the fractional seconds version yyyy-MM-dd hh:mm:ss.SSzzzzzzzzz.
> The code is fixed to handle this case as well.
Barry Lind
- Added new error message into errors.properties "postgresql.notsensitive"
This is used by jdbc2.ResultSet when a method is called that should
fetch the current value of a row from the database refreshRow() for
example.
- These methods no longer throw the not implemented but the new noupdate
error. This is in preparation for the Updateable ResultSet support
which will overide these methods by extending the existing class to
implement that functionality, but needed to show something other than
notimplemented:
moveToCurrentRow()
moveToInsertRow()
rowDeleted()
rowInserted()
all update*() methods, except those that took the column as a String
as they were already implemented to convert the String to an int.
- getFetchDirection() and setFetchDirection() now throws
"postgresql.notimp" as we only support one direction.
The CursorResultSet will overide this when its implemented.
- Created a new class under jdbc2 UpdateableResultSet which extends
ResultSet and overides the relevent update methods.
This allows us to implement them easily at a later date.
- In jdbc2.Connection, the following methods are now implemented:
createStatement(type,concurrency);
getTypeMap();
setTypeMap(Map);
- The JDBC2 type mapping scheme almost complete, just needs SQLInput &
SQLOutput to be implemented.
- Removed some Statement methods that somehow appeared in Connection.
- In jdbc2.Statement()
getResultSetConcurrency()
getResultSetType()
setResultSetConcurrency()
setResultSetType()
- Finally removed the old 6.5.x driver.
- These methods in org.postgresql.jdbc2.ResultSet are now implemented:
getBigDecimal(int) ie: without a scale (why did this get missed?)
getBlob(int)
getCharacterStream(int)
getConcurrency()
getDate(int,Calendar)
getFetchDirection()
getFetchSize()
getTime(int,Calendar)
getTimestamp(int,Calendar)
getType()
NB: Where int represents the column name, the associated version
taking a String were already implemented by calling the int
version.
- These methods no longer throw the not implemented but the new noupdate
error. This is in preparation for the Updateable ResultSet support
which will overide these methods by extending the existing class to
implement that functionality, but needed to show something other than
notimplemented:
cancelRowUpdates()
deleteRow()
- Added new error message into errors.properties "postgresql.noupdate"
This is used by jdbc2.ResultSet when an update method is called and
the ResultSet is not updateable. A new method notUpdateable() has been
added to that class to throw this exception, keeping the binary size
down.
- Added new error message into errors.properties "postgresql.psqlnotimp"
This is used instead of unimplemented when it's a feature in the
backend that is preventing this method from being implemented.
- Removed getKeysetSize() as its not part of the ResultSet API
Thu Jan 18 09:46:00 GMT 2001 peter@retep.org.uk
- Applied modified patch from Richard Bullington-McGuire
<rbulling@microstate.com>. I had to modify it as some of the code
patched now exists in different classes, and some of it actually
patched obsolete code.
Wed Jan 17 10:19:00 GMT 2001 peter@retep.org.uk
- Updated Implementation to include both ANT & JBuilder
- Updated README to reflect the changes since 7.0
- Created jdbc.jpr file which allows JBuilder to be used to edit the
source. JBuilder _CAN_NOT_ be used to compile. You must use ANT for
that. It's only to allow JBuilders syntax checking to improve the
drivers source. Refer to Implementation for more details
---------------------------------------------------------------------------
Attached is a set of patches for a couple of bugs dealing with
timestamps in JDBC.
Bug#1) Incorrect timestamp stored in DB if client timezone different
than DB.
timestamps in JDBC.
Bug#1) Incorrect timestamp stored in DB if client timezone different
than DB.
The buggy implementation of setTimestamp() in PreparedStatement simply
used the toString() method of the java.sql.Timestamp object to convert
to a string to send to the database. The format of this is yyyy-MM-dd
hh:mm:ss.SSS which doesn't include any timezone information. Therefore
the DB assumes its timezone since none is specified. That is OK if the
timezone of the client and server are the same, however if they are
different the wrong timestamp is received by the server. For example if
the client is running in timezone GMT and wants to send the timestamp
for noon to a server running in PST (GMT-8 hours), then the server will
receive 2000-01-12 12:00:00.0 and interprete it as 2000-01-12
12:00:00-08 which is 2000-01-12 04:00:00 in GMT. The fix is to send a
format to the server that includes the timezone offset. For simplicity
sake the fix uses a SimpleDateFormat object with its timezone set to GMT
so that '+00' can be used as the timezone for postgresql. This is done
as SimpleDateFormat doesn't support formating timezones in the way
postgresql expects.
Bug#2) Incorrect handling of partial seconds in getting timestamps from
the DB
When the SimpleDateFormat object parses a string with a format like
yyyy-MM-dd hh:mm:ss.SS it expects the fractional seconds to be three
decimal places (time precision in java is miliseconds = three decimal
places). This seems like a bug in java to me, but it is unlikely to be
fixed anytime soon, so the postgresql code needed modification to
support the java behaviour. So for example a string of '2000-01-12
12:00:00.12-08' coming from the database was being converted to a
timestamp object with a value of 2000-01-12 12:00:00.012GMT-08:00. The
fix was to check for a '.' in the string and if one is found append on
an extra zero to the fractional seconds part.
Bug#3) Performance problems
In fixing the above two bugs, I noticed some things that could be
improved. In PreparedStatement.setTimestamp(),
PreparedStatement.setDate(), ResultSet.getTimestamp(), and
ResultSet.getDate() these methods were creating a new SimpleDateFormat
object everytime they were called. To avoid this unnecessary object
creation overhead, I changed the code to use static variables for
keeping a single instance of the needed formating objects.
Also the code used the + operator for string concatenation. As everyone
should know this is very inefficient and the use of StringBuffers is
prefered.
I also did some cleanup in ResultSet.getTimestamp(). This method has
had multiple patches applied some of which resulted in code that was no
longer needed. For example the ISO timestamp format that postgresql
uses specifies the timezone as an offset like '-08'. Code was added at
one point to convert the postgresql format to the java one which is
GMT-08:00, however the old code was left around which did nothing. So
there was code that looked for yyyy-MM-dd hh:mm:sszzzzzzzzz and
yyyy-MM-dd hh:mm:sszzz. This second format would never be encountered
because zzz (i.e. -08) would be converted into the former (also note
that the SimpleDateFormat object treats zzzzzzzzz and zzz the same, the
number of z's does not matter).
There was another problem/fix mentioned on the email lists today by
mcannon@internet.com which is also fixed by this patch:
Bug#4) Fractional seconds lost when getting timestamp from the DB
A patch by Jan Thomea handled the case of yyyy-MM-dd hh:mm:sszzzzzzzzz
but not the fractional seconds version yyyy-MM-dd hh:mm:ss.SSzzzzzzzzz.
The code is fixed to handle this case as well.
Barry Lind
drivers.
The first fix fixes the PreparedStatement object to not allocate
unnecessary objects when converting native types to Stings. The old
code used the following format:
(new Integer(x)).toString()
whereas this can more efficiently be occompilshed by:
Integer.toString(x);
avoiding the unnecessary object creation.
The second fix is to release some resources on the close() of a
ResultSet. Currently the close() method on ResultSet is a noop. The
purpose of the close() method is to release resources when the ResultSet
is no longer needed. The fix is to free the tuples cached by the
ResultSet when it is closed (by clearing out the Vector object that
stores the tuples). This is important for my application, as I have a
cache of Statement objects that I reuse. Since the Statement object
maintains a reference to the ResultSet and the ResultSet kept references
to the old tuples, my cache was holding on to a lot of memory.
Barry Lind
edition of the driver did not compile. I have fixed both issues again. I have
attached the modified files to this email, maybe you can check them into the
repository. (Fixes are marked with //FIXME). Enterprise edition driver now
compiles and seems to work.
Jan Thomae
absolute. It also makes it more compliant with the interface
specification in Sun's documentation;
1. absolute(0) should throw an exception.
2. absolute(>num-records) should set the current row to after the last
record in addition to returning false.
3. absolute(<num-records) should set the current row to before the first
record in addition to returning false.
These operations in the existing code just return false and don't change
current_row.
These changes required a minor change to relative(int) since it calls
absolute(int)
The attached patch is against the cvs repository tree as of this morning.
Also, who is in charge of maintaining the jdbc driver? I'm working on
getArray for the jdbc2 driver, but it's going to require three more
classes to be added to the driver, and thus three more source files
in the repository. Is there someone I can contact directly to ask about
this?
Travis Bauer | CS Grad Student | IU |www.cs.indiana.edu/~trbauer
It addresses three issues:
1. The problem with ResultSet's interface specifying 1-based indexing was
not quite fixed in 7.0.2. absolute would stop the user form moving to the
first record (record 0 internally).
2. Absolute did not set current_row
3. For field.mod=-1, GetObject would try to return numeric values with a
precision of around 65000. Now GetObject detects when field.mod==-1, and
passes that as the scale to getBigDecimal. getBigDecimal detects when a
-1 is passed and simply does not scale the value returned. You still get
the correct value back, it simply does not tweak the precision.
I'm working off of a source tree I just checked out from the
repository. The diff is based on what was in the repository about ten
minutes ago.
----------------------------------------------------------------
Travis Bauer | CS Grad Student | IU |www.cs.indiana.edu/~trbauer
----------------------------------------------------------------
Fixed Statement, so that the update count is valid when an SQL DELETE operation is done.
While fixing the update count, made it easier to get the OID of the last insert as well. Example is in example/basic.java
