Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintainer #84

Closed
guenhter opened this issue Sep 30, 2024 · 9 comments
Closed

Maintainer #84

guenhter opened this issue Sep 30, 2024 · 9 comments

Comments

@guenhter
Copy link
Contributor

@willhallonline

It seems that you could use some support. We really need an up to data version of angular and therefore a well maintained ansible docker image. If you are willing, me and a colleague would help out here.
If you are not interested and don't have time for this repo any more, we will start our own repo.

@jaydrogers
Copy link

@guenhter: What issues are you having with this image?

@jaydrogers
Copy link

Looking further into this, I could see where there are areas of concern. 2.16 is reaching end of life in November and there hasn't been any update on 2.17 support (#83 opened by @eliasfrigard).

Screenshot 2024-10-01 at 14 45 14@2x

As an open source maintainer myself, I totally understand. @willhallonline has a life to live. His personal life is number 1 priority. I am beyond grateful for @willhallonline's contributions and all of his hard work. I would never expect an open source maintainer to even respond if life is just too busy. He's done all of this for us and has received nothing in return. It's makers like this who help us keep the digital industry moving forward.

I really don't want to have the community start the discussion of a fork unless if @willhallonline for some reason cannot respond.

I've noticed in my own projects things are generally better when there are processes for the community to provide their own contributions. This reduces stress and dependencies on a single maintainer.

Moving forward

@willhallonline: If you are busy -- just let us know. No offense taken and we totally get it. If there is a process you'd like to have in place for other maintainers to help out, let us know what that looks like. If adding outside maintainers is too much of a hassle -- no problem. We can always work together and reassemble on our own.

Just let us know what works best for you. Again, I cannot express in a simple GitHub post how grateful I am for your work. This project is a huge foundational brick in my other open source projects and my concepts would've never been possible without this project.

@christ-re
Copy link
Contributor

Looking further into this, I could see where there are areas of concern. 2.16 is reaching end of life in November and there hasn't been any update on 2.17 support (#83 opened by @eliasfrigard).

Exactly and i had already a Pull request open to support 2.17 since June. (#80)
It was merged today. But still it was 3 1/2 months for not that much of changes.

I think we can all understand that life is number 1 and its @willhallonline's decision. But we still also need a kind of updated versions of this wonderful open-source project.

And I can only agree with you @jaydrogers and @guenhter that if any help is needed, we can help.

@willhallonline
Copy link
Owner

Hey, I totally appreciate all of the kind words in this. It is a bit of a funny project that I have been running for around 6 years which mainly started for my own uses, I've probably done 3/4 major re-architectures of the project on the whole and probably made hundreds (maybe near thousands) or containers in that time.

Over time, the hardest thing has always been coping with what I would probably say are unintended consequences, but largely attributed to getting the right versions of everything running on the right base image - I have lost countless hours trying to balance OS, Python, Ansible, Ansible-Core, Ansible-lint together so that it appears seamless. Also, now when I actually send things out it is still making around 60 containers every tag, but if something goes wrong it is normally around 60 minutes to build and so I have often context shifted away from it.

I am totally up for more contributions. Maybe someone wants to drop some rules in around contributing as a PR? I can do it, but trying to balance this around doing lots of other things.

@willhallonline
Copy link
Owner

Also, I would love it if you wanted to put in that you are using the project in anything else? As I am totally separated from end users, as in, I actually use the containers infrequently so never actually see anything from end users, how they use the container images or what the usage stats are? Is there any that I can easily drop (is anyone still using Alpine 3.16)??

@jaydrogers
Copy link

Thanks for getting back @willhallonline!

My biggest suggestions

Based on my perspective, this is what I would suggest:

Convert CI/CD to GitHub Actions

Everything seems to run on a GitLab CI, but it's difficult for others to contribute since we don't have access to any debugging or knowledge of the CI environment.

We just converted our PHP Docker images to use RunsOn with AWS. In our PHP project, I was running about 40-60 builds at a time and it took forever because of the 20 concurrent job limit.

We have faster builds, but it does cost about $10-30/mo to run that CI environment (with how much we run builds). You aren't charged as much if the environment sits idle (~$0.20/day)

Get Ansible 2.17 Ready

It seems like you're already making progress on getting this resolved #83

I think that alone will reduce a lot of the fears in the posts above.

Explore how to run the app as a different user

One thing that I need to test and figure out (specifically for my use case, but could be very helpful for others) is allowing Ansible to run as an unprivileged user and under dynamic UID & GIDs. The current images run as root by default, which can lead to very strange permission behaviors for people.

This item is more of a note for myself on what I need to explore with this image, but there is a chance it could turn into some big Dockerfile changes depending on my tests. I'm hoping it's just a simple documentation thing that I can gladly contribute 😃

Make all images multi-arch

Supporting multi-arch across the board is very important for compatibility. The disadvantage it really adds a ton of time to the builds, but it does have a huge payback on developer experience as more and more people are switching to ARM processors.

Answering your other questions

Here's a few comments to answer your other questions

Projects that use this image

The project we heavily use this image on is Spin: https://serversideup.net/open-source/spin/

It's an open source tool that allows people to spin up and manage their applications with Docker. We use this Ansible image to allow people to run our Spin Ansible Collection against any server of their choice. They can run something as simple as spin provision and this image will connect to their server and configure it as a Docker Swarm cluster for deployments.

I can't express how grateful and important this image is to our open source work ❤️

Contribution Guidelines

A lot of this direction comes from the Lead Maintainer's expectations. GitHub links these from their docs:

In our projects, I don't really have any rules. It's basically notes for myself and if others want to join 😃

Getting stats on pulls

You could put deprecation warnings in older images and stop updating them if you wanted, but I would definitely have some sort of policy explaining what you support. We do this with a SECURITY.md file.

Docker does have an Insights and Analytics but you need to apply for the Docker Open Source Program.

Thanks again

I just want to re-emphasize my gratitude for your work. My suggestions above are simply just suggestions. Don't feel any pressure to take on what you don't want to support. I just wanted to share my notes of what I had on my end.

Let us know your thoughts moving forward and what you need help with.

@willhallonline
Copy link
Owner

Wow. That is a lot to unpack and thanks for writing such a lot. What I am going to try to do is split this down into different issues and then I can try to deal with them one-by-one.

  1. Move to Github Actions Migrate CI and Build to GitHub Actions #86
    a. The reason it is on GitLab CI is two-fold. Firstly, it pre-dates the existence of GitHub actions. Secondly, I have been a major
    GitLab user and therefore haven't seen the value in moving for my own personal things. However, happy to consider this as I
    don't think the re-write is too considerable.

  2. Get Ansible 2.17 ready Add support for ansible 2.17 #80
    a. I think that these should be available now.

  3. Run as separate user Run container as separate user rather than root #87
    a. Yep. We totally should run as a different user. Probably needs that changing across all current Dockerfiles. My interest is normally how this would affect running in CI, as I see that as it's main utility as a container, however, 100% would be nice to have.

  4. Make Images multi-arch
    a. Not sure if should make this as another issue. latest is multi-arch (or at least has support for arm64) and I did quite a lot of basically failing work relating to cross-compiling around a year ago. The challenge was that it was taking soooo looonnnggg to make them for arm that often the runners would time out. I did try running my local machine as a secondary gitlab runner, however, that took remembering to use it.
    b. Maybe make a suggestion of the images you would like to be on arm and I will just do those, I don't think it is worthwhile to do all images on arm (my preference would just be for alpine).

  5. Projects that use this Projects that use this tool #88
    a. Your project looks cool. I now have one known user ;).

  6. Contributing Guidelines Add CONTRIBUTING.md #89
    a. I'll try to look and copy one that I like. Funnily enough, I do a bunch of this in my day job, but, I don't know, something like the cobblers children having no shoes (https://english.stackexchange.com/questions/159004/the-cobblers-children-have-no-shoes).

  7. Add a SECURITY.md Add SECURITY.md to document security deprecation #90
    a. OK, I will look into it.
    b. Not too sure how I would show deprecation warnings. I did delete hundreds/thousands of containers from Docker Hub around a year ago that were really old and therefore full of vulnerabilities, but not sure what is best for this.

  8. Apply to Docker Open Source Progam Apply to Docker Open Source Program #91
    a. We'll see how that goes. I was in the GitLab open source program for a while, but was missing some parts and so dropped off and didn't see much value in re-joining.

@jaydrogers
Copy link

jaydrogers commented Oct 5, 2024

So I think I have it just about working: https://github.com/serversideup/docker-ansible

Have no pressure merging any of this, but here's what I did:

GitHub Actions

My GitHub Actions takes about 8 minutes to build 36 different images: https://github.com/serversideup/docker-ansible/actions/runs/11190826459

The images upload to DockerHub and GitHub Actions. I separated Ansible and Ansible Core into two separate repos:

Everything is based off of a single Dockerfile

You can see my Dockerfile supports Debian and Alpine. Build arguments are created and tags are determined by what's set in ansible-versions.yml. This allows me to process logic of what is "latest" and what package dependencies need to be insalled on a per operating system basis.

So far I only have these operating systems supported:

  • Debian Bullseye
  • Alpine 3.20

The images are all based off of Python's official Docker images. This allows me to build many versions of Ansible using different versions of Python.

All images are multi-arch

I have linux/amd64 and linux/arm64 set for all images.

Latest Ansible Versions Are Automatically Pulled From PyPi

If I define a version in my Ansible Variations, then the minor patch is automatically pulled from PyPi's API. So If I pass it 2.17 for ansible-core, it automatically will grab 2.17.4 as the latest version.

Remaining ToDos

  • I need to get Ansible Lint installed on all the images
  • I am thinking of NOT including Mitogen because of some mixed reviews
  • I need to figure out the changing of the UID and GID for dropping permissions

Don't feel any pressure to use what I made. I just wanted to share what I learned today. If you like it, I can contribute a PR I can once I stabilize things and figure this out 😆

Any thoughts or feedback, let me know.

@jaydrogers
Copy link

✨ Just another update

I have the following features available at https://github.com/serversideup/docker-ansible/

  • 🐧 Debian and Alpine - Choose your OS
  • 🐍 Built on official Python images - Choose your Python version
  • 🔒 Unprivileged user - Choose to run as root or an unprivileged user
  • 📌 Pinned Ansible Version - Set your Ansible version down to the patch version
  • 🔧 Customize your "run as" user - Customize the username to run as
  • 🔑 Set your own PUID and PGID - Have the PUID and PGID match your host user
  • 📦 DockerHub and GitHub Container Registry - Choose where you'd like to pull your image from
  • 🤖 Multi-architecture - Every image ships with x86_64 and arm64 architectures

Privilege management

The hardest part with Docker containers is running things with proper privileges (especially when you're dealing with things like SSH keys and configs).

Default unprivileged user

I created an ansible user that will be used for "unprivileged" execution: https://github.com/serversideup/docker-ansible/blob/73f83299098735af83948cbb2a64a8a3cb63a03b/src/Dockerfile#L19-L41

Changing who to execute as

From there, I let the entrypoint handle the logic. Ansible with suexec or gosu (depending on if they are running Alpine or Debian): https://github.com/serversideup/docker-ansible/blob/main/src/rootfs/entrypoint.sh

People can also set their PUID and PGID via environment variables to ensure mounted volumes are accessed using the same IDs.

⏩ Moving forward

Merging my changes as a PR would be a MAJOR change, but I don't expect you to do that. I wanted to share my source code with you since your image deeply inspired my concept to build Spin. Feel free to look through my source code and adapt as you see fit for this project!

If you need any help or have any questions, let me know. Thanks @willhallonline!

@guenhter guenhter closed this as completed Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants