ORM or SQL: Are We There Yet?

A decade ago I was first introduced to servlets. It was a revelation. Prior to that I (like most people) had been doing CGI scripts (in Perl and even C and C++). Suddenly we had something that persisted between requests. Combined with JDBC you had a pretty powerful platform.

In 2001 or so began the madness that was the Java world's love affair with EJBs. I came into it at the EJB 1.1 stage with EJB 2.0 just on the horizon. A consultant at the time decided it was a good idea to write a logging service as an entity bean. It took roughly half a second to write a log message!

This began my questioning of and ultimate departure from the EJB. Going through 2002, cracks were surfacing. It became widely accepted that EJBs weren't suitable for high throughput applications. The pro-EJB crowd argued that EJBs weren't suitable for all applications but that shouldn't mean you dismiss the technology.

Experience has taught me that when someone wheels out this argument of selective observation that it's a huge red flag.

Apart from stateless sessions beans, which are a fairly cheap and effective way of getting distributed transactions in a clustered J2EE environment, EJB (pre-3.0) was basically a bad idea.

By this stage J2EE had started to fracture. Soon would come Spring, which would change the Java server landscape forever. The other big change was Hibernate.

Hibernate became the poster-child for post-EJB OO fanatics. I was late to this particular party. I'd gone back to doing plain SQL and was happy. Less than two years ago I was forced to learn JPA and I gave it a fair shake of the stick, I really did.

The relational-object divide has been a divisive issue for many years. Jeff Atwood went as far as saying Object-Relational Mapping is the Vietnam of Computer Science.

I am a steadfast believer that abstractions are leaky. And an object model on top of a relational model is an abstraction. To jam this into the Java/J2EE world, apparently mechanisms like load-time weaving were required.

Now I believe that those behind such changes were (and are) well-intentioned but, as the quote goes, the road to hell is paved with good intentions. I see the problem like this: imagine a Cartesian graph with one axis representing complexity and the other representing the completeness of the abstraction. The more "complete" it is, the less leaky it is. I picture such a graph would produce an hyperbola. JDBC is simple because it's not much of an abstraction. Hibernate and other JPA providers are incredibly complex because they are attempting to be complete.

There are two corollaries you can draw from this analogy that I think fit:

  1. To achieve a perfect abstraction the solution would be infinitely complex (ie impossible); and
  2. There is a "sweet spot" in the middle where a little abstraction has a large reward but, beyond a certain point, there is a law of diminishing returns.

I also believe that every developer should be educated in relational algebra, comfortable with databases and proficient in SQL. Just like any Java Web developer should understand the servlets API before they can truly truly appreciate and properly leverage a higher-order MVC framework like Spring MVC or Struts 2. Ultimately, you still need to know something about how HTTP works. The same applies to SQL.

This is why I believe the effort to create the "perfect" object model for relational databases is futile and ten plus years later we're still not there. And you know what? I don't think we'll be there in another ten years either. If anything, the issue will become a non-issue as the traditional relational database will be replaced by "slacker" databases, persistent caches or whatever comes after that.

Until then, perhaps we should stop trying to fit a round peg into a square hole. SQL just works, it's not that hard and your application will be less complex as a result. That can only be good.

14 comments:

Casper Bang said...

+1. I have a really hard time seeing why colleagues love this complex model of writing sub-optimal, type-unsafe queries inside Java annotations with no way for me to test them ad-hoc.

No doubt you've also run into people claiming the ORM problem is solved? I respectfully disagree with them. The closest I've seen this "Vietnam of CS" (to quote Ted Neward) solved is on the .NET stack with LINQ. Sadly we'll never see such expressiveness on the conservative Java stack.

SqAR said...

You don't know WebObjects and the Enterprise Objects Framework (EOF). All that happened years before servlets, EJBs, J2EE, Spring, Hibernate, JPA and all the other buzz-wordy stuff was created. I don't want to repeat myself so google for it.

William Shields said...

To paraphrase Joel: yes but "nobody" used it (and by "nobody" I mean less than 20,000 programmers). Some algorithms on notinoal "binary adding machines" were developed 200+ years ago but the "real" age of computing didn't really start until Wolrd War 2 (Enigma code-cracking), shortly afterwards (1948, Univac) or arguably not until mass commercialization (1970s for mainframes, 1980s for Macs/PCs).

These arguments always come up and theres nearly always something or someone in tech who really did something first but until it reaches sufficient crtical mass, they're largely just footnotes.

Anonymous said...
This comment has been removed by a blog administrator.
Art Vandalay said...

While I agree with almost all of your premises I ultimately disagree with your conclusion of "stop trying" ORM. If someone is planning to design an application effectively with objects and persist it to a relational database, ORM is inevitable; the only question at that point is whether the coder will invest valuable time in manually writing the bridge every...single...time, or develop expertise in a specialized tool and API.

It's definitely a hopeless round peg-square hole situation - one of those "hard" problems that can't have a generalized solution with no trade-offs - but that just means the coder must take on the responsibility of plugging the leaks in the abstraction as the leaks crop up. That is, coding some vital queries in straight (DB-specific) SQL, and sometimes persisting complicated collections of objects (circular references?) with a custom strategy.

So no disagreement here on the need for coders to know SQL (and the relevant database) well, for when the abstraction inevitably leaks.

Oziel Joze said...

The thing is, if you want a truly OO application you need a orm framework, and by the way using plain sql to create a aplication just sucks.

Lots of time spent on doing the same thing over and over again.

If you want to use plain sql always, you shouldnt be using java anyway, go to php and youll be fine.

To me your explanation smells like someone who dont truly understand any orm framework.

Anonymous said...

@Oziel,

With respect, this is a pretty typical response. "If you want true OO, then you have to do X".

I feel that's a statement that causes quite a bit of harm overall... add layers of complexity to do something simple, just because the simple thing might take a little longer (even if the simple thing might give you a better result).

I personally feel that if you're working properly with a database, you're probably NOT "doing the same thing over and over again". A enterprise-grade database is NOT just a "place to store your objects", and to relegate it to such a status is moronic. You *should* be thinking about how you store your data, and leveraging the database tier accordingly, instead of handing it all to some pile of code that thinks it knows best. We do that enough already.

"Using plain SQL to create an application just sucks" ?!?! Please tell me you are kidding.

To me, your explanation smells like someone who dont [sic] truly understand what SQL can do, that your best ORM would take many more CPU cycles to accomplish.

mnuttall said...

@Anonymous - There is more to an enterprise application's data then just a RDBMS.

Whethere you feel it or not, if you don't have an abstraction layer outside the db you WILL be doing things over and over.

To me, your comments are from someone who trully doesn't understand what a good ORM can do.

mnuttall said...

Dang fingers - should be "Whether".

Oziel said...

@Anonymous

What i meant was that ONLY using plain sql to develop a enterprise application was the problem, cus i think that a combination of both is the perfect solution, and im not kiddind.

By the way if your application is rather specialized and specific use sql to get the most of the database, but if you try to develop a complex app filled with business logic than a orm might be the perfect fit.

Good orm frameworks can be very optimised on a per query basis, if that still is not enough use sql. Oracle for example provides many addon methods to toplink jpa orm that uses specific and advanced features of their database.

ben said...

This is exactly why I stopped using Hibernate and started using iBatis, its primarily sql with a nicer api than JDBC.

Anonymous said...

@ben

I have always had a love/hate relationship with Hibernate. I have been using it for years now, though, I think, I was kindof "forced" to because of the huge hype around it and the company I work using it also.

It does its job yes but there are aspects in its architecture that I don't like - mostly transparent persistence. That is because of the things that happen implicitly and also because of the overhead it takes to make all the magic work. In addition and contrary to many "architects" I prefer writing SQL myself.

Unfortunately hibernate won the hype war and also achived to turn itself into THE official standard thus hindering precious mindshare from less popular tools like iBatis.

vinothkumar said...

What is this truly OO Application ? lol ..
there is no such thing as true OO Application..

OO , ORM , Design patterns all boils down to cutting cost of maintenance .

hire two junior engineers to maintain code written using Plain SQL .. pay them 30K each..

hire a genius senior engineer to maintain the O/RM based code .. pay him 100K ..

Lukas Eder said...

Every tool has its purpose and vision. I have created http://jooq.sourceforge.net to wrap SQL in Java objects. But it is still SQL!

jOOQ has basic ORM features, but it mainly focuses on the things that I guess most developers need most, when trying to find the best ORM for their needs:

* code generation
* variable binding (that's a pain in JDBC)
* SQL syntax abstraction (to prevent syntax errors)

But often they go too far and provide so much abstraction, you wouldn't think they're running against an RDBMS. On the other hand, you chose an RDBMS precisely because

* it is a robust data source
* SQL can do many good, performant things (nested selects, unions, complex joins, etc). Often ORM's cannot do these things.
* you can handle transactions and sessions yourself
* you have UDT's and stored procedures

jOOQ addresses exactly these points. It will perform as well as JDBC, but without the pain.

Post a Comment