MyOwnDB v2: tech lessons applied

As briefly announced previously, a new version of MyOwnDB has been deployed. This new version is actually a complete rewrite based on a different technology stack, moving from Ruby on Rails to F# and Websharper.

The first version of MyOwnDB was launched in 2006, with the first snapshot in archive.org of the website dating from 18 May 2006. If the initial technology stack served us well, it was time to assess the choices made nearly 20 years ago. Here are the lessons learned, explaining some choices made in the development of MyOwnDB v2 (applicable to the development of MyOwnDB, YMMV).

A dynamically typed language makes it harder to apply significant updates to the code.

There seems to be two camps in programming circles: those choosing dynamically typed languages, and those choosing statically typed languages (for which the compiler catches more errors before the core actually runs in production). MyOwnDB tries to make it easy for its users to manage their data, but this requires to complexity under the hood. And although there were extensive tests, this approach felt cumbersome and unreliable with a dynamically typed language. That led to MyOwnDB stagnating for quite some years. And that’s why a statically typed language was chosen for the new version. And it proved a very good choice, with multiple refactoring made easier or even possible by the static types. My feeling is that during a refactoring, compiler errors are faster to get and are more precisely pointing you in the right direction than tests runtime errors.

The ecosystem matters

This is not in opposition to the previous version, as Ruby on Rails has a tremendous ecosystem. But initially the preferred language choice to develop MyOwnDB v2 was Ocaml, a statically typed functional language with good performance and some facilities to ease the use of Funtional Programming compared to Haskell, which is purely functional (eg regarding side effects). But OCaml’s ecosystem was lacking (IIRC from 2019: there was no mature library to query postgresql, and even the installation of Ocaml failed for obscure reason on an Ubuntu system). Others have tried harder to use Ocaml, but eventually made the same choice as MyOwnDB: switch to F#, a cousin of OCaml developed by Microsoft on dotnet.

Running on Dotnet, F# gives access to all libraries developed for dotnet. There was not need to search for the right library to interact with postgresql: Npgsql is an easy choice. Other examples are:

• NGettext, which helped make MyOwnDB multilingual from the start
• Legivel, a yaml parser that helped reuse the fixtures from the Ruby on Rails app
• Humanizer, helping to manipulate strings, also in the context of loading fixtures
• Hangfire, for handling background processing
• Mailkit for mail sending from the app
• Evolve for handling database migrations

Of course you can find equivalents for other languages, but F# seems to strike the right balance, being an advanced, approachable ML-family language with a extensive ecosystem.

Splitting front-end and backend code makes it harder

MyOwnDB was initially developed in the hey-days of Ajax, and it tried to exploit the technology to provide a better experience. But this led to the use of some javascript libs on the client, talking to the Ruby on Rails server. Not only did this bring the need to work in two distinct languages, it also required to handle the communication between those two ends. That’s why a better solution was sought for the new version of MyOwnDB.

The goal was to find a way to develop both backend and frontend in F#, and if possible, abstract away all communication between them. And that’s exactly what’s brought by WebSharper. WebSharper lets you mark F# functions to be made available on the client side, compiling it to Javascript to be run on the client. Of course you can’t compile code that use server-side-only libraries, but this didn’t appear to be a big problem. For example, the NGettext library, used to manage translation on the server side, is not available to be compiled as Javascript. But the fact that WebSharper lets you use Javascript libraries from your F# code stil allowed us to have an uniform handling of translations across both client and server sides.

Communications between the client and the server are also very easy: mark the server-side function you want to make callable from the client with the [<Rpc>] annotation. Just doing that makes that function available from your client-side F#, with arguments transparently serialised, and communication errors reported by raising exceptions. Again some limitations may apply as your arguments types need to be serialisable, but this never caused trouble in our case.

And that’s without even mentioning Websharper’s Reactive Vars, which additionally eased the development of a maintainable dynamic UI.

Extensive tests are a must

Some dynamic languages proponents claim that testing is a sufficient substitute to static typing. Having developed extensively with a statically typed language, I disagree (though I won’t try to convince anyone not sharing this opinion). But the use of a statically typed language does not make tests optional! Both are complementary to develop a reliable solution. That’s why MyOwnDB has an extensive tests suite, both for the backend library code as for the frontend code.

If you want to be free, be it from the start!

If your plan is to publish your code under an open source license, do it from the start. The code for the second version of MyOwnDB was shareis from the initial commit. This will prevent you to commit something you wouldn’t want to share, which you would have to cleanup first when the time has come to publish your project under an open source license. Not doing it from the start makes it more difficult. And the more you wait, the more the difficulty probably grows. There are exceptions of course (Ruby on Rails comes to mind), but why not make it easier for you to hold your word and reach your goal of publishing an open source project?

Simpler is usually more maintainable

In the first version of MyOwnDB, some bleeding-edge choices appeared to be a burden on the longer term. The best example of that is the choice to store file attachments in S3. It was the early days of the cloud, and using S3 was technically interesting, and gave MyOwnDB some visibility, but in the longer term the benefits were rather small. The current MyOwnDB made some choices that are not exactly hype-compliant:

• Docker Swarm over Kubernetes: it is not expected that MyOwnDB will need the features and extreme scalability brought by Kubernetes, and Docker Swarm seems to cover all needs while still bringing a ton of value in the ease of deployment
• monolithic vs micro-services: the MyOwnDB is clearly a monolithic application, with no need of a micro-services architecture

This does not mean that bold choices are to be avoided at all cost. Bold choices should also be done, when they bring a clear added value, like F# and WebSharper do for MyOwnDB.