ToroDB Stampede is an open source middleware that speaks the MongoDB protocol and replicates data live from MongoDB to an RDBMS. Data is transformed on-the-fly from a document model into a pure relational model, not a single flattened table. Schema is created automatically, there’s no need to design any tables beforehand. ToroDB is built 100% with Java8, and this talk is about its internals, how some significant problems are solved and what techniques and frameworks are used and why, such as:
- Network I/O with Netty, off-heap buffers and memory consumption.
- How D2R (Document-to-Relational) transformation is performed.
- How DDL changes are performed, isolation challenges and why some more naive considered alternatives do not work.
- How CUD (Create, Update, Delete) changes are transformed and applied into batches, as they are consumed from the MongoDB replication stream. Use of Akka Streams.
- How ToroDB Stampede deals with sharding.
- Docker integration into maven for fully automated unit and integration testing scenarios, including clusters of servers, failover, etc. JUnit 5 taken to the extreme!
Come and join this session to see how a high-performance database system can be built with Java. And of course contribute your own ideas and expertise, we’re open source and welcoming PRs!
Alvaro Hernandez, ToroDBahachete
Álvaro is a 38-year-old Spanish entrepreneur. Founder and CEO at 8Kdata, a database R&D company. Spends most of his time working on the ToroDB project, the first NoSQL-on-SQL database, a MongoDB-compatible database that runs on top of an RDBMS and is written in Java. He is a passionate Java software developer and open source advocate. Álvaro is a member of JavaSpecialists.eu and jCrete.org, and a frequent speaker at international conferences.