🎧 Kaizen 11 - S3 R2 B2 D2 - Aug. 11, 2023

[gerhard]

🎧 This is now available as changelog.com/friends/10

See all improvements in this 🐙 GitHub repository: Kaizen 10 ... 11

---

This contains all things related to the next Kaizen episode.

Share your thoughts via comments below:

• Is there something in changelog.com that you, our dear listener, would like us to tackle?
• Is there something that you would like to help out with?
• Anything that we - @adamstac @jerodsanto - want to improve?

This is a rough timeline that we have in mind (red line is today):

gantt
    title 🎧 Kaizen 11 timeline
    dateFormat  YYYY-MM-DD
    axisFormat %b %e, %Y

    section DISCUSS
    ✍️ Add your comments below :2023-07-01, 38d

    section CONTRIBUTE
    💻 ALL CONTRIBUTIONS WELCOME 👕 :2023-07-01, 38d

    section RECORD


    🎙 Record :milestone, 2023-08-08
    ✂️ Edit :2023-08-08, 3d
    🚀 SHIP IT! :milestone, 2023-08-11

    section NEXT
    Kick-off Kaizen 12 :2023-08-11, 14d

Loading

---

🎙 Proposed topics

• Phoenix v1.7 upgrade
• PostgreSQL v15 upgrade
• Transactional Email delivery via Oban
• Migrate from AWS S3 to Cloudflare R2
  • Extended Storage Lens
• https://status.changelog.com
• Migration to Fly apps v2 (a.k.a. machines) is now complete
• A more resilient CI setup ✌️☝️
  • Fly.io
    • https://community.fly.io/t/fdaa4556-3-wireguard-gateway-intermittent-failure/14597
  • GitHub Actions fallback
  • K8s (Talos) 💪
• Honeycomb SLOs - all two of them!
• Fastly.com LHR POP latency increased by 50% on June 20th, 2023
  • Fixed on July 25th, 2023

Honeycomb SLOs

• 95% of the time, podcast feeds should be served within 1s (last 30 days)
• regardless whether cached or uncached
• breakdown by podcast
• p99 - from 1s e.g. GoTime to 11s for Changelog & 25s for Master

• 98% of all responses are 2xx or 3xx (last 30 days)
• let's dig into this one
• what is
  • 403 /images/podcasts/master-original-3922a5ce5ede81d973adc3cd618f7e64.png
  • 403 /uploads/podcast/534/changelog-interviews-534.mp3
  • FWIW "Changelog Interviews" pods do not play at all on the website #457 (comment)

What would be a better SLO for you?

More than 2 SLOs? 🤔

• errors by library.name

Fastly.com LHR POP network latency increase

Started June 20th. Fixed July 25th.

---

What changed so far? Kaizen.10 ... NOW

• Upgrade to Phoenix 1.7
• Deliver all mailers through Oban
• Add status.changelog.com #470
• Ensure app deploys work as expected on Fly.io machines #475
• Make our ship_it.yml GHA workflow resilient #476
• Upgrade PostgreSQL from v14.1 to v15.3 #477
• Run Dagger Engine as a Fly Machine (no more Docker) #471
• Migrate from AWS S3 to Cloudflare R2 #468

What else is coming up?

• Install and configure Oban Web #472
• Add new caching variant for multi-node clustering #451
• https://middleware.io/
• 1Password Service Accounts

---

This is a follow-up to:

• 🎧 Kaizen 10 - The best pipeline ever™️ - Jun. 2, 2023 #452
• 🎧 Kaizen 9 - Embracing change 🌟 - Mar. 2, 2023 #440
• 🎧 Kaizen 8 - 24 improvements & a lot more - Nov. 23, 2022 #433

[jerodsanto]

PR incoming that I'll need your help with @gerhard: I signed us up for Oban Web and we will need to get it all set up, which includes doing some fancy work with the deployment pipeline. Install directions are here:

https://getoban.pro/docs/web/2.9.7/installation.html

I think this could be a good Kaizen 11 topic as well since it is related to the Oban mailers task (assuming we get 'er done in time)

[gerhard]

Sounds great! Adding it to the list.

[gerhard]

FYI https://elixirforum.com/t/using-dagger-elixir-sdk-to-refactor-a-script/56748

[jerodsanto]

Kaizen 11 has shipped! 🚀 https://changelog.com/friends/10