| Commit message (Expand) | Author | Age |
|---|
| * | Rework loading revision data•••These changes were motivated by switching to a mechanism of loading data that
isn't dependent on the big advisory lock that prevents more than one revision
from being processed at a time.
Since INSERT ... RETURNING id; is used, this can block if another transaction
inserts the same data, and then cause an error when that transaction
commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle
the case when the INSERT doesn't return an id since the other transaction has
inserted it.
This commit rewrites insert-missing-data-and-return-all-ids to do as described
above, as well as being more efficient in how existing data is detected and to
use more vectors. Other utilities for inserting data are added as well.
| Christopher Baines | 2024-12-09 |
| * | Start trying to handle GC happening while processing revisions | Christopher Baines | 2024-08-12 |
| * | Get the test suite working again | Christopher Baines | 2024-08-07 |
| * | Update tests | Christopher Baines | 2024-07-26 |
| * | Support setting environment variables in the inferior•••When processing jobs, this is mostly to allow setting GUIX_DOWNLOAD_METHODS.
| Christopher Baines | 2024-06-24 |
| * | Further change job store connection handling•••The guix-dameon WAL is still growing excessively, so avoid doing anything with
the long running store connection except registering temporary roots.
| Christopher Baines | 2024-04-25 |
| * | Fixup tests | Christopher Baines | 2024-01-18 |
| * | Use fibers when processing new revisions•••Just have one fiber at the moment, but this will enable using fibers for
parallelism in the future.
Fibers seemed to cause problems with the logging setup, which was a bit odd in
the first place. So move logging to the parent process which is better anyway.
| Christopher Baines | 2023-11-05 |
| * | Set current-guix-package when computing system test derivations•••This is a bit ugly, but might speed up computing derivations for system tests.
| Christopher Baines | 2023-02-28 |
| * | Allow skipping processing system tests•••Generating system test derivations are difficult, since you generally need to
do potentially expensive builds for the system you're generating the system
tests for. You might not want to disable grafts for instance because you might
be trying to test whatever the test is testing in the context of grafts being
enabled.
I'm looking at skipping the system tests on data.guix.gnu.org, because they're
not used and quite expensive to compute.
| Christopher Baines | 2023-02-08 |
| * | Add missing test description | Christopher Baines | 2022-11-11 |
| * | Partition the package_derivations_by_guix_revision_range table•••And create a proper git_branches table in the process.
I'm hoping this will help with slow deletions from the
package_derivations_by_guix_revision_range table in the case where there are
lots of branches, since it'll separate the data for one branch from another.
These migrations will remove the existing data, so
rebuild-package-derivations-table will currently need manually running to
regenerate it.
| Christopher Baines | 2022-05-23 |
| * | Close the load revision inferior prior to inserting data•••This means that the lock can be acquired after closing the inferior, freeing
the large amount of memory that the inferior process is probably using.
| Christopher Baines | 2022-03-11 |
| * | Address a few issues in the load new guix revision tests | Christopher Baines | 2022-03-02 |
| * | Track package replacements•••Start at least looking for package replacements, and storing the
details (particularly the derivation). I'm looking at doing this so that build
servers using the Guix Data Service can build these derivations.
| Christopher Baines | 2021-07-11 |
| * | Fetch the list of system values from the database•••This removes the need to hardcode some values in the code.
| Christopher Baines | 2021-04-23 |
| * | Add powerpc64le-linux as a supported system•••This should not be necessary, but currently the database isn't used to find
out the valid system values.
| Christopher Baines | 2021-02-12 |
| * | Fix indentation in (tests model-license-set) | Christopher Baines | 2021-01-02 |
| * | Improve select-job-for-commit•••It now returns #f or a list, rather than the empty list or a nested list.
| Christopher Baines | 2020-10-09 |
| * | Use the git repository fetch with authentication value | Christopher Baines | 2020-10-07 |
| * | Change the locale codeset representation•••From the normalized one, to the one actually contained within glibc. Recent
versions of glibc also contain symlinks linking the normalized codeset to the
locales with the .UTF-8 ending, but older ones do not.
Maybe handling codeset normalisation for queries would be good, but the locale
values ending in .UTF-8 are more compatible and allow the code to be
simplified. For querying, maybe there should be a locales table which handles
different representations.
| Christopher Baines | 2020-09-26 |
| * | Fix the test-model-package-metadata test•••Signed-off-by: Christopher Baines <mail@cbaines.net>
| Danjela Lura | 2020-06-19 |
| * | Fix the test-model-package test•••Signed-off-by: Christopher Baines <mail@cbaines.net>
| Danjela Lura | 2020-06-19 |
| * | Fix lint checker tests•••Signed-off-by: Christopher Baines <mail@cbaines.net>
| Danjela Lura | 2020-06-07 |
| * | Update the valid-systems test | Christopher Baines | 2020-04-20 |
| * | Do some more mocking in the jobs-load-new-guix-revision tests•••To help with getting them running on Guile 3.
| Christopher Baines | 2020-02-27 |
| * | Remove unused mock-inferior import | Christopher Baines | 2020-02-27 |
| * | Don't use the (tests driver) module•••As I forget why I even added it. I'm sure it was to try and make test failures
more understandable, but I'm not sure it worked.
| Christopher Baines | 2020-02-27 |
| * | Add a missing import | Christopher Baines | 2020-02-27 |
| * | Adapt some license related code to work without mock in the tests•••With Guile 3, there's a potential for mock to work in even fewer
circumstances. So, adapt the code to enable writing the tests without mock.
| Christopher Baines | 2020-02-24 |
| * | Check that the test database is being used in all the tests | Christopher Baines | 2020-02-24 |
| * | Clear a couple of tables in the load-new-guix-revision tests•••As entries from incomplete previous runs can effect the test, so clean out any
old data before the test begins.
| Christopher Baines | 2020-02-24 |
| * | Fake the store connection in the tests•••So that the tests don't require a store connection.
| Christopher Baines | 2020-02-14 |
| * | Use a more long lived store connection for loading data•••As this will enable registering temporary roots, to avoid store items being
garbage collected.
| Christopher Baines | 2020-02-13 |
| * | Start storing channel instance derivations•••These are the ones that relate to Guix pull.
| Christopher Baines | 2020-02-11 |
| * | Update the load-new-revision-job tests | Christopher Baines | 2020-02-11 |
| * | Make valid-systems quicker•••In the future, it would be good to pull this from the database again, but in
some way which is really quick.
| Christopher Baines | 2020-01-05 |
| * | Insert channel news entries when loading new guix revisions•••If the channel-news-for-commit function is present.
| Christopher Baines | 2019-11-21 |
| * | Refactor channel handling in the load-new-guix-revision module•••To allow for passing the channel to channel-news-for-commit.
| Christopher Baines | 2019-11-21 |
| * | Improve the package and package-metadata modules•••Add tests around the package module, extract out the use of the
inferior-package record assessors so that they aren't part of the tests, and
switch across the package-metadata module to use
insert-missing-data-and-return-all-ids.
| Christopher Baines | 2019-09-05 |
| * | Put some comments in the mock-inferior module•••About mocking the record file accessors not working :(
| Christopher Baines | 2019-09-05 |
| * | Start handling ids as numbers, rather than strings•••squee, returns all data as strings, and expects strings as inputs to
queries. So, keeping the ids as strings was easy initially, but it means that
you can't tell from the type whether it should be quoted, or not...
Therefore, handle ids as strings, converting them to numbers when they're
fetched from the database, and back to strings as part of the queries.
| Christopher Baines | 2019-09-05 |
| * | Store lint warnings in the database•••This commit adds the relevant tables and code to store lint warnings in the
database.
Currently, only lint checkers which don't require access to the network will
be run, as this allows the processing to happen without network access. Also,
this functionality won't work in older versions of Guix which don't expose the
lint warnings in a compatible way.
| Christopher Baines | 2019-09-01 |
| * | Avoid erroring when processing emails again•••These changes allow processing emails again, and just creating job and branch
entries where data is missing.
| Christopher Baines | 2019-08-05 |
| * | Remove test data after the tests have ran•••The load-new-guix-revision tests aren't transactional, so truncate some tables
after running the tests.
| Christopher Baines | 2019-08-05 |
| * | Fix some duplicated values in tables•••The licenses table, along with the package_metadata table had duplicate
values. This could happen as the unique constraints on those tables didn't
properly account for the nullable fields.
The duplicates in those tables also affected the license_sets, packages,
package_derivations tables in a similar way. Finally, the
guix_revision_package_derivations table was also affected.
This commit adds a migration to fix the data, as well as the constraints. THe
code to populate the licenses and package_metadata tables is also updated.
| Christopher Baines | 2019-08-04 |
| * | Switch to processing jobs in parallel•••This should speed up processing new revisions, reduce latency between finding
out about new revisions and processing them, as well as help manage memory
usage, by processing each job in a process that then exits.
| Christopher Baines | 2019-07-12 |
| * | Fix the name for the jobs-load-new-guix-revision test module | Christopher Baines | 2019-07-12 |
| * | Associate a name with database connections•••This helps when working out which connection to the database is doing what.
| Christopher Baines | 2019-07-12 |
| * | Add a new test to cover some of the loading new revisions code | Christopher Baines | 2019-06-02 |