Wednesday, April 6, 2011

A Co-Relational model?

Reading the following interesting article A Co-Relational Model of Data for Large Shared Data Banks left me with several questions...

Isn't the co-relational model is an overcomplication of things? Isn't the noSQL algebra is just relational algebra without the Cartesian product operator? I mean that relational algebra without the Cartesian product operator can be used for noSQL databases, except graph databases. Primitive operators of relational algebra such as projection, selection and set operators are pretty much scalable. Cartesian product is not. Authors mentioned that property of the Cartesian product in the example with WWW database. In order to emulate Cartesian product in noSQL developers have to use external tools such as Map Reduce which has nothing to do with databases.

Why wouldn’t the noSQL providers use SQL as a starting point? Take SQL standard, relax transactional requirements from ACID to BASE, add functions for searching in columns, add a note that execution of queries with Cartesian product may be extremely slow (with Map Reduce or other parallelization technology. Let the database vendors compete in that field). I recommend book “Data-Intensive Text Processing with MapReduce” by Jimmy Lin and Chris Dyer book for further reading on how to implement relational joins with MapReduce. DML and DDL will be changed substantially, though. Drop or relax relational constraints, different table definition etc. As for coSQL, it looks to me like it introduces Codasyl-syle queries instead of relational algebra.

Monday, March 14, 2011

Do we really need Model Driven Engineering or Domain-Specific languages?

There is a mathematical notion of program complexity that called Kolmogorov complexity.
Informally, it states that complexity of a program is more than complexity of a task the program solves.
Some logical implications from Kolmogorov complexity:
- Programming must take time proportional to Kolmogorov complexity of the task.
- A program cannot be fully designed in a time that significantly less than time spent on programming of the task.

So why do we need one more layer of languages to program the task?
I don't think that methodologies like Model Driven Engineering or Domain-specific modeling will dramatically decrease time needed to develop a solution to any problem. Complexity of the problem remains constant whichever tool you use.

Wednesday, February 16, 2011

Numbers Everyone Should Know

One damn useful slide from Jeff Dean talk. Slide appears on video at 0:59:00

Operation ns
L1 cache reference 0.5
Branch misprediction 5
L2 cache reference 7
Mutex lock/unlock 25
Main memory reference 100
Compress 1k w/cheap compression algorithm 3,000
Send 2K bytes over 1Gbps network 20,000
Read 1MB sequentially from memory 250,000
Round trip within same datacenter 500,000
Disk seek 10,000,000
Read 1MB sequentially from disk 20,000,000
Send packet CA->Netherlands->CA 150,000,000