It was quite an experience to partake in the Jessie release – and also a rather long “Saturday”. This post is mostly a time line of how I spent my release day with doing the actual release. I have glossed over some details – the post is long enough without these. 🙂
We started out at 8 (UTC) with a final “dinstall” run, which took nearly 2 hours. It was going to take longer, but we decided to skip the synchronisation to “coccia.debian.org” (the server hosting the DD-accessible mirror of release.debian.org).
The release itself started with the FTP masters renaming the aliases of Squeeze, Wheezy and Jessie to oldoldstable, oldstable and stable respectively. While they worked, the release team reviewed and double checked their work. After an hour (~11), the FTP masters reported that the stable releases were ready for the final review and the SRMs signed the relevant “Release” files.
Then the FTP masters pushed the stable releases to our CD build server, where Steve McIntyre started building the installation images. While Steve started with the CDs, the FTP masters and the release team continued with creating a suite for Stretch. On the FTP/release side, we finished shortly before 12:30. At this point, our last ETA from Steve suggested that the installation media would take another 11 and a half hours to complete. We could have opened for mirror synchronisation then, but we decided to wait for the installation media.
At 12:30, there was a long “intermission” for the release team in the release process. That was an excellent time to improve some of our tools, but that is for another post. 🙂
We slowly started to resume around 22:20, where we tried to figure out when to open for the mirror synchronisation to time it with the installation media. We agreed to start the mirror sync at 23:00 despite the installation media not being completely done then. They followed half an hour later, when Steve reported that the last CD was complete.
At this point, “all” that was left was to update the website and send out the press announcement. Sadly, we were hit by some (minor) issues then. First, I had underestimated the work involved in updating the website. Secondly, we had no one online at the time to trigger an “out of band” rebuild of the website. Steve and I spent an hour and a half solving website issues (like arm64 and ppc64el not being listed as a part of the release). Unsurprisingly, I decided to expand our the “release checklist” to be slightly more verbose on this particular topic.
My “Saturday” had passed its 16th hour, when I thought we had fixed all the website issues (of course, I would be wrong) and we would now just be waiting for the an automatic rebuild. I was tempted to just punt it and go to bed, when Paul Wise rejoined us at about 01:25. He quickly got up to speed and offered to take care of the rest. An offer I thankfully accepted and I checked out 15 minutes later at 01:40 UTC.
That more or less covers the Jessie release day from my PoV. After a bit of reflection inside the release team, we have found several points where we can improve the process. This part certainly deserves its own post as well, which will also give us some time to flesh out some of the ideas a bit more. 🙂