Vulcan - Gpre, JDBC, and a Modest Proposal

From Jim Starkey on the Firebird Development List 25th January 2004

I think it's time to talk about removing all dependencies on gpre and gradually phase it out. It isn't feasible or desirable to drop it altogether, but eliminating all internal use would be a step forward.

The original idea behind gpre (and its DEC precursors) was that ease of use came from tight language integration. A tight language integration meant a preprocessor. And a preprecessor freed the API to be object/message based and make remote servers perform well, along with BLR so the compilation work could be done at preprocessor times, reducing the amount of runtime work. This is all well and good, but GDML lost and SQL won.

DSQL was designed as a quickie for a specific client, then productized. Rather than inventing an API, I took the DB2 API lock, stock, and barrel figuring no one was going to argue with IBM. The API stinks. Borland moved the DSQL implementation from the client to the server without fixing anything. I'm moving DSQL from the Y-valve into the engine where it belongs. One of these days we can stop calling it DSQL and start calling it just SQL.

The biggest problem with DSQL is that the API is close to unusable. Anyone who could love SQLDAs could love MPE/XL.

I suggest that Firebird maintain the existing API, but select/design a new API to gradually become the primary product API. I recommend that a C++ binding be adopted. I implemented one in the OdbcJdbc project. A representative piece of code:

PreparedStatement *statement = connection->prepareStatement (
  "select * from RDB$RELATIONS where RDB$RELATION_ID=?");
statement->setInt (1, relationId);
ResultSet *resultSet = statement->executeQuery();
while (resultSet->next())
 const char *relationName = resultSet->getString ("RDB$RELATION_NAME");
 printf ("Got one: %s\n", relationName);

The interface is semantically and almost syntactically identical to Java JDBC.

There would be, over time, three implementations of the interface. One, inside the engine, would be used in modules to replace the existing pre-processed modules (met, dfw, ini, etc). Once complete, the engine could be built without preprocessor.

The second implementation would be a client side library implemented with the existing DSQL API. This already exists in OdbcJdbc and could be rolled out with next to no effort.

The third implementation would be a full integration of the interface into the plumbing -- Y-valve, remote protocol, and engine. The third implementation, however, would be completely transparent. The client library would probe the Y-valve to see if it supported the new calls. If so, it would attempt to route a call through the plumbing. If this failed, it would do local emulation. This strategy would allow a network to gradually shift over to the new API without breaking anything. After a couple of reasons and due deliberation, the client library could disappear completely in favor of full architectural support of the new API.

A problem not addressed is non-OO languages that couldn't use JDBC directly. A translation layer with handles would probably make most sense, though the old API would remain in place for the indefinite future.

Over time, the internal implementation and the internal DSQL could invert, with DSQL layered on JDBC rather than vice versa, backed up by a pure SQL engine running in parallel to the legacy BLR engine.

None of this is going to happen as part of Vulcan, though I must admit that dumping the internal gpre-ed modules is mighty tempting. But I do think that this is a good time to discuss it.