Commit 60de553a authored by sajolida's avatar sajolida

Merge remote-tracking branch 'origin/master'

parents a69d9271 fd4d170b
......@@ -32,11 +32,160 @@ Everything in this report is public.
# B. Improve our quality assurance process
## B.4. Freezable APT repository
The work we have done has been reviewed, merged into the main Tails
development branch, and successfully used while preparing Tails
2.4~rc1. Unsurprisingly, we had to fix a couple small bugs that
earlier testing had not discovered, but all in all we're very
satisfied by how the whole thing work: it has been very solid and
performed pretty well so far. We are proud to point to the first ever
tagged snapshot, that contains only the set of packages needed for
building Tails 2.4~rc1, and the corresponding source code:
Now, let's dive into the details:
- B.4.3. Centralize and merge the list of needed packages
As [[explained previously|contribute/reports/SponsorS/2015/2016_03#index4h2]],
the original definition of this deliverable doesn't make sense
anymore, so here we are reporting about what now replaces it:
* Allow storing APT snapshots longer than the default when needed:
the code was reviewed, merged, and successfully used in production
while preparing Tails 2.4~rc1, so this is completed.
* Freeze and unfreeze the APT snapshots used by a branch when
needed: the code and corresponding documentation were reviewed,
merged, and used in production, so this is completed.
So we're happy to report that deliverable B.4.3 has been completed
in May.
- B.4.5. Implement processes and tools for importing and freezing those packages ([[!tails_ticket 6299]], [[!tails_ticket 6296]])
As [[said last month|contribute/reports/SponsorS/2015/2016_04]], the
last remaining bits here are about handling some consequences on
this system:
* Garbage collection of APT repository snapshots: this was deployed
in production and works fine.
* Manage a very custom configuration for `apt-cacher-ng`: this was
reviewed, merged, and used in production since then.
* Manage `reprepro`'s database growth: we checked the actual data
in our production environment and realized that there is actually
no problem to be solved here; since we have enabled garbage
collection, the database has not grown at all.
- Miscellaneous follow-ups
We have submitted upstream three branches that improve the Puppet
module we use to manage `reprepro` in ways that made it compatible
with the needs of our freezable APT repository.
By the end of July, we will also do some polishing in various areas:
* Polish a bit the design documentation for the entire setup
([[!tails_ticket 11447]]).
* If needed, write helper tools for freeze exceptions
([[!tails_ticket 11448]]).
* Investigate a weird issue we have identified, when a package is
not removed from our time-based APT snapshots, while it should be
([[!tails_ticket 11496]]).
# C. Scale our infrastructure
## C.1. Change in depth the infrastructure of our pool of mirrors
XXX: u, please review
The new mirror pool is now used by Tails Upgrader, by users who
download Tails without using our Download And Verification Extension
for Firefox (aka. DAVE), for any download that is not supported by
DAVE (e.g. release candidates), and for downloads started from a web
browser that has JavaScript disabled. So, in summary two of the use
cases of this work are covered already, and only the "downloading with
DAVE" use case is left to complete.
- C.1.2. Write & audit the code that makes the redirection decision from our website ([[!tails_ticket 8639]], [[!tails_ticket 8640]], [[!tails_ticket 11109]])
* `mirror-dispatcher.js`: we are still waiting for the auditor to do
a final security review.
* Download And Verification Extension for Firefox: we have made some
progress on the implementation, and coordinated with the person
who will do the code review to ensure he will be available when we
need it. XXX: u, please update/complete this part.
- C.1.4. Communicate with each mirror operator to adapt their configuration ([[!tails_ticket 8635]], [[!tails_ticket 11079]])
This deliverable is completed:
* All mirrors have now implemented the changes we requested.
* We have sent a call for mirrors to a number of fast mirror
operators, and we already have 7 more mirrors. We will pursue this
effort in June, even though we have already reached the goals we
had set: we expected to have at least 30 mirrors in the pool once
the new infrastructure was ready, and 35 mirrors 3 months later,
and we already have 36 active mirrors as of May 31.
- C.1.6. Adjust download documentation to point to the mirror pool dispatcher's URL ([[!tails_ticket 8642]], [[!tails_ticket 11329]], [[!tails_ticket 10295]])
This was deployed to production: all links pointing to our mirror
pool now use the new redirection system.
So, this deliverable is now completed.
- C.1.7. Adjust update-description files for incremental upgrades ([[!tails_ticket 11123]])
We have adjusted the code of Tails Upgrader to use the new mirror
pool. This code has been merged, and is now used by Tails Upgrader
in production (starting with Tails 2.4~rc1), so this deliverable is
completed as well.
- C.1.8. Clean up the remainers of the old mirror pool setup ([[!tails_ticket 8643]], [[!tails_ticket 11284]])
This is now only blocked by the work that is in progress on DAVE
## C.4. Maintain our already existing services
XXX: bertagaz, please review/complete
- C.4.6. Administer our services upto milestone VI
We kept on answering the requests from the community and taking care
of security updates.
We noticed that old Puppet reports were not cleaned up as they
should on our infrastructure, so we fixed this and submitted a merge
request to the Puppet module we use to manage… Puppet itself
([[!tails_ticket 11468]]).
We noticed that our four newest virtual machines used to
continuously run our automated test suite on all ISO images built by
our Jenkins instance (B.2) did not reboot as intended between test
suite runs. We investigated the root cause of the problem, and fixed
it ([[!tails_ticket 11467]]).
We ported everything that made sense to, in our Puppet
infrastructure, to use [Hiera](
Not only this simplified a lot how we manage systems, but more
importantly, this allowed us to release quite a bit more of our
Puppet code. This is part of our strategy to treat infrastructure as
code, and to enable more people to contribute to it without needing
any special credentials.
# D. Migration to Debian Jessie
We streamlined email reporting from failed cronjobs across our
infrastructure, to ensure we don't miss problems.
We did lots of refactoring and miscellaneous clean ups in our Puppet
code. Sprint cleaning!
# E. Release management
......@@ -8,13 +8,18 @@ This is about [[!tails_ticket 5926]].
1. design documentation: [[!tails_ticket 11447]]
2. [[!tails_ticket 11445]]: handle ever-growing `references.db`, aka.
[[!debbug 823629]]: if
`references.db` doesn't fit in the memory disk cache, then at
least our GC process gets very slow); the visible consequence
would be: long periods of heavy disk read, and much slower
snapshots expiration process; so we added an Icinga2 check on
the file size itself. When this problem occurs, our options are:
2. misc: see subtasks of [[!tails_ticket 5926]]
# Bonus for later
## Handle handle ever-growing `references.db`
This is about [[!debbug 823629]]:: if `references.db` doesn't fit in
the memory disk cache, then at least our GC process gets very slow);
the visible consequence would be: long periods of heavy disk read, and
much slower snapshots expiration process; so we added an Icinga2 check
on the file size itself. When this problem occurs, our options are:
* add more RAM to the VM if that's still feasible and reasonable
(likely not)
* reset the whole `debian` repository to an empty state (simple
......@@ -30,10 +35,6 @@ This is about [[!tails_ticket 5926]].
works on small databases, but on our big `debian` the file
doesn't shrink
3. misc: see subtasks of [[!tails_ticket 5926]]
# Bonus for later
## Miscellaneous
If the chosen mirroring/snapshoting tool supported re-using the Debian
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment