guix/qa-frontpage

1. Local Development
2. Information flow
3. Ideas

-- mode: org --

This service is intended to assist with quality assurance within Guix. That is maintaining and improving the quality of many aspects of Guix, like packages for example.

1. Local Development

Use the guix-dev.scm file to provide the dependencies.

guix environment -l guix-dev.scm

Then run the following commands:

./bootstrap.sh
./configure
make

After that, you can start the service via:

./pre-inst-env scripts/guix-qa-frontpage

Expect pages to load slowly since the QA frontpage won't have up to date cached data to use.

2. Information flow

Figure 1: Diagram describing how information moves between different components

3. Ideas

3.1. For branches

3.1.1. TODO Show broken packages

3.1.2. TODO Show broken system tests

TODO Submit builds for branch system tests

3.1.3. TODO Show broken fixed output package derivations

3.1.4. TODO Show lint warnings

3.1.5. Branch comparisons

TODO Show broken system tests
TODO Show broken fixed output package derivations
TODO Show new lint warnings

3.1.6. DONE Show package reproducibility statistics

This will provide a better URL and faster page load times compared to directly going to data.guix.gnu.org or data.qa.guix.gnu.org.

3.2. For patches

3.2.1. TODO Better handle an influx of new bugs

Currently when this happens, I think QA may delete a bunch of branches that now fall outside the latest issues it looks at, only to re-process these once they reappear as the new issues are dealt with (either by being merged or closed).

3.2.2. TODO Show and describe the overall status on the page for an issue

3.2.3. TODO Improve display of lint warning changes

TODO Say which package the lint warnings apply to

3.2.4. TODO Show when there are comments/messages on the issue

To highlight when there is discussion that might need reading before merging the patch.

3.2.5. TODO Show changes with system tests

TODO Submit builds for system tests when affected by patches

3.2.6. TODO Extend testing for patches

TODO Cross-compilation from x86_64-linux

This can be enabled, but data might often be lacking due to lack of build resources for bordeaux.guix.gnu.org
TODO powerpc64le-linux

The one powerpc64le-linux build machine for bordeaux.guix.gnu.org isn't always on (this could be fixed by changing the hosting). Anyway, some always available build machine for powerpc64le-linux is probably needed before extending patch testing.
TODO i586-gnu

Builds are now happening on bordeaux.guix.gnu.org, once it's caught up, enabling i586-gnu should be possible. There might be issues with a lack of build resources (like x86₆₄-linux).
TODO Test reproducibility

This depends on getting more data on builds back to the data service.

Submit multiple builds for the same derivation.

3.2.7. DONE Improve patches page

DONE Add more statuses
DONE Support filtering by status

3.2.8. TODO Run licensecheck on package sources

And highlight changes or when this doesn't match what's declared in the Guix package.

3.3. QA Ecosystem

3.3.1. data.qa.guix.gnu.org

TODO Move away from Chris renting beid (the machine it runs on)
TODO Make processing revisions faster and require less resources
TODO Support accepting build reports/buildinfo

This can include the hashes of the output and be factored in to the assessment of reproducibility. This will allow using the results of multiple builds per build server, rather than just being able to look at a single substitute.

3.3.2. Monitoring and observability

TODO Move away from Chris hosting Prometheus/Grafana for this purpose

Either to Prometheus packaged for Guix and run on Guix machines, or some other approach.

3.3.3. Keeping Guix packages up to date

TODO Automated patch submission for package updates

Using guix refresh. This way the manual work is reduced, and it's easier to just apply the package updates that have been tested.
1. TODO Speed up QA for patches
  
  As this would result in more patch series, QA (mostly the data service) will need speeding up a lot to make this feasible.

3.3.4. Facilitating people without commit access reviewing patches

This might help speed up getting patches merged, and get more people involved in the process. Maybe the QA Frontpage has a part to play in helping this to happen.

DONE Add mark patches as "reviewed-looks-good" feature

3.4. Reproducible builds

3.4.1. TODO Improve issue display, show severity and separate by status

3.4.2. TODO Record and expose percentage of packages that build reproducibly

Compute the percentage, ignoring packages where the status is unknown. Also ignore systems with little data available.

Expose in Prometheus metrics, and record in the database. This could be backfilled by looking through the older data service data.

Table of Contents