(quotes from an article)
Make your team’s deploys as boring as hell and stop stressing about it.
start with a good foundation: have good tests. Don’t skimp out on this, because it impacts everything else down the line.
How fast should fast really be? GitHub’s tests generally ran within 2-3 minutes while I was there.
[Feature Flags let you] test code in the real world with real data and make sure things work and “feel right”
You should also be using pull requests, merge requests, or other code review to keep track of discussion on the code you’re introducing.
Your branch is your responsibility … If it breaks, you fix it
Start reviews early and often. … If you can open a code review with imaginary code to gauge interest in the interface, for example, those twenty minutes spent doing that and getting told “no, let’s not do this” is far preferable than blowing two weeks on that full implementation instead.
build software using quick, tiny branches and pull requests. Make them small enough to where it’s easy for someone to drop in and review your pull in a couple minutes or less. If you build massive branches, it will take a massive amount of time for someone else to review your work, and that leads to a general slow-down with the pace of development. … This is where those feature flags from earlier come into play.
Again, the smaller the diff, the more boring, straightforward, and stress-free your deploys become.
Branch deploys: When you’re ready to deploy your new code, you should always deploy your branch before merging. Always.
Whatever you have on your master branch … should be noted as being the absolute reflection of what is on production. In other words, you can always be sure that your master branch is “good” and is a known state where the software isn’t breaking. … If you merge your branch first into master and then deploy master, you no longer have an easy way to determining what your good, known state is without doing an icky rollback in version control.
The benefit of some type of audit trail for your deployments is basically what you’d expect: you’d be able to find out who deployed what to where and when. Every now and then you’ll run into problems that don’t manifest themselves until hours, days, or weeks after deployment, and being able to jump back and tie it to a specific code change can save you a lot of time.
Deploy locking is basically what you’d expect it to be: locking production so that only one person can deploy code at a time. There’s many ways to do this, but the important part is that you make this visible.
A deploy queue has a couple parts: 1) if there’s a wait, add your name to the end of the list, and 2) allow for people to cut the line (sometimes Really Important Deploys Need To Happen Right This Minute and you need to allow for that).
The final bit of housework that’s required is the cleanup. … One approach I liked to take was to prepare two pull requests: one that toggles the feature flag (i.e., ships the feature to everyone), and one that cleans up and removes all the excess code you introduced. … You should celebrate this internally, too: it’s the final sign that your coworker has successfully finished what they were working on. And everyone likes it when a diff is almost entirely red. Removing code is fun.
I only get emotional about two things: a moving photo of a Golden Retriever leaning with her best friend on top of a hill overlooking an ocean looking towards a beautiful sunset, and deployment workflows.
At the end of the day, I care about two things: how my coworkers are feeling, and how good the product I’m working on is. Everything else stems from those two aspects for me.
Spend some time and get your deploys to be as boring, straightforward, and stress-free as possible. It’ll pay off.