Replacing docker-compose with Nix for development

en/blog.md

@@ -1,13 +1,12 @@
 ---
 order: 15
 feed:
-  enable: false
+  enable: true
   title: NixOS Asia Blog
 ---
 
 # Blog
 
->[!warning] WIP
-> We don't have any posts yet. Checkout our [[tutorial]] in the meanwhile.
-
-An RSS feed will be made available soon (cf. [this bug](https://github.com/srid/emanote/issues/490)).
+```query {.timeline}
+path:./*
+```

en/blog.yaml

@@ -0,0 +1,9 @@
+template:
+  sidebar:
+    enable: false
+  uptree:
+    enable: true
+  breadcrumbs:
+    enable: true
+  base:
+    containerClass: container mx-auto max-w-7xl

en/blog/replacing-docker-compose.md

@@ -0,0 +1,122 @@
+---
+author: shivaraj
+date: 2023-03-05
+page:
+  image: replacing-docker-compose/ny-services-flake.png
+---
+
+# Replacing docker-compose with Nix for development
+
+Ever since I first started using [[nix|Nix]] for #[[dev|development]], I have enjoyed the [[why-dev|simplicity of setup]]: `nix develop`, make the code change and see it work. That's all well and good, but when your project keeps growing, you need to depend on external services like databases, message brokers, etc. And then a quick search will tell you that [docker](https://www.docker.com/) is the way to go. You include it, [add one more step](https://github.com/nammayatri/nammayatri/tree/f056bb994fbf9adefa454319032ca35c34ea65bc/Backend#other-tools) in the setup guide, increasing the barrier to entry for new contributors. Not to forget, eating up all my system resources on my not so powerful, company provided, macOS[^native-macos].
+
+This, along with the fact that we can provide one command to do a lot of **cool things** (which you will see, as you continue to read), made us want to replace [docker-compose](https://docs.docker.com/compose/) with Nix in [Nammayatri](https://github.com/nammayatri/nammayatri) (to keep it simple, I will refer to it as NY from now on).
+
+> [!note] Nammayatri
+> [NY](https://nammayatri.in) is an open-source auto rickshaw booking platform, based in India.
+
+[^native-macos]: There is an [initiative to run containers natively on macOS](https://github.com/macOScontainers/homebrew-formula), but it is still in alpha and [requires a lot of additional steps](https://github.com/macOScontainers/homebrew-formula?tab=readme-ov-file#installation) to setup. One such step is [disabling SIP](https://developer.apple.com/documentation/security/disabling_and_enabling_system_integrity_protection#3599244), which a lot of company monitored devices might not be allowed to do. 😕
+
+{#what-does-it-take}
+
+## What does it take?
+
+Turns out, there is not a lot of things that we need to do: we need to be able to run services natively, across platforms (so that my macOS doesn't drain its battery running a database), and integrate with the existing [flake.nix](https://github.com/nammayatri/nammayatri/blob/main/flake.nix) (to avoid an extra step in the setup guide).
+
+Lucky for us, NixOS already has a clean way to run services natively. We can use that as inspiration and build on top it.
+
+{#nixos-services}
+
+## NixOS services
+
+Running services in NixOS is a breeze. For example, [running a PostgreSQL Database](https://nixos.wiki/wiki/PostgreSQL) is as simple as adding one line to your configuration:
+
+```nix
+{
+  services.postgresql.enable = true;
+}
+```
+
+This starts the database natively, without the need for a container. This is great, only if we could extend this to run on macOS and other Linux distributions. Also, the configuration above is system-wide, and we are looking for a way to run services on a per-project basis.
+
+{#nixos-like-services}
+
+## NixOS-like services
+
+That's a relief, we are now closer to our goal. We now know what we want, and there are a few more things that needs to be added to the list:
+
+- Allow for running multiple instances of the same service (NY uses multiple instances of PostgreSQL and Redis).
+- Along with services being project specific, their data should also be project specific.
+
+These were the exact problems #[[services-flake]] was designed to solve. Along with running services natively, it also [integrates with your project's `flake.nix`](https://community.flake.parts/services-flake/start).
+
+>[!info] How are processes managed in services-flake?
+> In NixOS they are managed by [systemd](https://en.wikipedia.org/wiki/Systemd). In services-flake, we use [process-compose](https://github.com/F1bonacc1/process-compose) whose configuration is managed by [process-compose-flake](https://community.flake.parts/process-compose-flake).
+
+{#let-s-get-started}
+
+## Let's get started
+
+Now that we have all the answers. It's time to replace [docker-compose in NY](https://github.com/nammayatri/nammayatri/blob/f056bb994fbf9adefa454319032ca35c34ea65bc/Backend/nix/arion-configuration.nix) with [[services-flake]]. We will only look at a few services to keep it simple, for more details, see [PR](https://github.com/nammayatri/nammayatri/pull/3718).
+
+:::{.center}
+![[ny-services-flake.png]]
+:::
+
+{#postgresql}
+
+### PostgreSQL
+
+NY uses about 3 instances of postgresql databases.
+
+One of them is [exported by passetto](https://github.com/nammayatri/passetto/blob/nixify/process-compose.nix) (passetto is a haskell application that encrypts data before storing in postgres), and using it looks like:
+
+```nix
+{
+  services.passetto.enable = true;
+}
+```
+
+By using [[modules|module system]], we can hide the implementation details and only expose the `passetto` service to the user, and use it as above.
+
+The other two instances are used by the [postgres-with-replica module](https://github.com/nammayatri/nammayatri/blob/ccab8da607cfd8d4e9f7d28b55b83e22eec1af9b/Backend/nix/services/postgres-with-replica.nix). This module starts two services (`primary` and `replica` database) and a [pg-basebackup](https://www.postgresql.org/docs/current/app-pgbasebackup.html) process (to sync `replica` with `primary` during init). For the user it is:
+
+```nix
+{
+  services.postgres-with-replica.enable = true;
+}
+```
+
+{#redis}
+
+### Redis
+
+NY uses [Redis](https://redis.io/) as a cache and clustered version of it as a Key-Value database. Redis service comprises of a single node, while the clustered version has 6 nodes (3 master and 3 replica). Adding them to the project is as simple as:
+
+```nix
+{
+  services.redis.enable = true;
+  services.redis-cluster.enable = true;
+}
+```
+
+{#cool-things}
+
+## Cool things
+
+By not depending on docker anymore, we can now run the entire NY backend with one command and its all defined in a [single place](https://github.com/nammayatri/nammayatri/blob/ccab8da607cfd8d4e9f7d28b55b83e22eec1af9b/Backend/nix/services/nammayatri.nix). 
+
+That is not all, we can also share NY backend module to do much more, like defining [load-test](https://github.com/nammayatri/nammayatri/blob/ccab8da607cfd8d4e9f7d28b55b83e22eec1af9b/Backend/load-test/default.nix) config and run it in CI/local, again, with one command. Here, we take the module to run entire NY stack, and then extend it to add a bunch of load-test processes, before bringing the whole thing to an end (as the load-test ends).
+
+This is how running them looks like:
+
+```sh
+# Run load-test
+nix run github:nammayatri/nammayatri#load-test-dev
+
+# Run the entire backend
+nix run github:nammayatri/nammayatri#run-mobility-stack-nix
+```
+
+## Up next
+
+Sharing [[services-flake]] modules deserves a separate post, so we will talk more about this in the next post.

en/event.md

@@ -1,4 +1,6 @@
+```
 
+```
 # Events
 
 We host events to bring together the community. Our primary venue is in [Bangalore].

en/services-flake.md

@@ -0,0 +1,4 @@
+
+# services-flake
+
+[services-flake](https://community.flake.parts/services-flake) provides declarative, composable, and reproducible services for [[dev|Nix development environment]], and is based on #[[flake-parts|flake-parts]]. Enabling users to have [[nixos|NixOS]]-like service on [[macos|MacOS]] and Linux.