| Commit message (Expand) | Author | Age |
|---|
| * | Make build_servers.id just generated by default•••As to make it easier to set explicitly.
| Christopher Baines | 2024-12-15 |
| * | Add utility functions for configuring the database | Christopher Baines | 2024-12-15 |
| * | Use knots•••A library of extracted Guile Fibers patterns and utilities.
| Christopher Baines | 2024-12-15 |
| * | Add a comment | Christopher Baines | 2024-12-09 |
| * | Log differently when using the load-new-guix-revision-inserts lock | Christopher Baines | 2024-12-09 |
| * | Avoid getting the load-new-guix-revision-inserts lock twice | Christopher Baines | 2024-12-09 |
| * | Fix hash display for formatted derivations | Christopher Baines | 2024-12-09 |
| * | Improve some query formatting | Christopher Baines | 2024-12-09 |
| * | Fix wrong number of values from channel-derivations-by-system->guix-store-item | Christopher Baines | 2024-12-09 |
| * | Rework loading revision data•••These changes were motivated by switching to a mechanism of loading data that
isn't dependent on the big advisory lock that prevents more than one revision
from being processed at a time.
Since INSERT ... RETURNING id; is used, this can block if another transaction
inserts the same data, and then cause an error when that transaction
commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle
the case when the INSERT doesn't return an id since the other transaction has
inserted it.
This commit rewrites insert-missing-data-and-return-all-ids to do as described
above, as well as being more efficient in how existing data is detected and to
use more vectors. Other utilities for inserting data are added as well.
| Christopher Baines | 2024-12-09 |
| * | Improve null handling | Christopher Baines | 2024-12-01 |
| * | Handle conflicts in insert-missing-data-and-return-all-ids | Christopher Baines | 2024-12-01 |
| * | Memoize field-can-be-null? | Christopher Baines | 2024-11-29 |
| * | Handle conflicts when inserting lint warning message sets | Christopher Baines | 2024-11-29 |
| * | Use insert-missing-data-and-return-all-ids for locations | Christopher Baines | 2024-11-29 |
| * | Add in call-with-worker-thread to try to avoid sort problems | Christopher Baines | 2024-11-29 |
| * | Move log message | Christopher Baines | 2024-11-29 |
| * | Don't lookup #f in the hash table | Christopher Baines | 2024-11-29 |
| * | Insert guix_revision_package_derivation_distribution_counts later•••As this should speed up processing revisions.
| Christopher Baines | 2024-11-26 |
| * | Don't crash when linters crash•••This is currently happening because some linters try to evaluate parts of
packages for cross-building to aarch64-linux-gnu, but not all packages support
that and some crash in this case.
I'm not quite sure what the correct behaviour should be, but maybe the data
service needs to try and handle these crashes rather than not processing the
entire revision.
| Christopher Baines | 2024-11-26 |
| * | Restore calling update-derivation-ids-hash-table!•••In derivation-file-names->derivation-ids. This is important for performance
since it avoids reading derivations with an entry already in the database.
| Christopher Baines | 2024-11-25 |
| * | Change to logging the filtered derivations•••As that's what's relevant here.
| Christopher Baines | 2024-11-25 |
| * | Stop queries being able to insert systems | Christopher Baines | 2024-11-13 |
| * | Allow using the heap profiler in the inferior | Christopher Baines | 2024-11-08 |
| * | More efficiently create some vectors•••And instrument the inferior memory after creating the gds-inferior-packages
vector.
| Christopher Baines | 2024-11-08 |
| * | Add a heap-profiler module•••Taken from a Guile mailing list post.
| Christopher Baines | 2024-11-08 |
| * | Do more when cleaning inferiors•••Not sure if this actually helps, but it might do.
| Christopher Baines | 2024-11-08 |
| * | Add log-tag when inserting channel instance derivations | Christopher Baines | 2024-11-08 |
| * | Fix git-commit-exists? | Christopher Baines | 2024-11-08 |
| * | Fix sorting license sets | Christopher Baines | 2024-11-08 |
| * | Fix starting with an empty database | Christopher Baines | 2024-11-08 |
| * | Have the background resource pool users be very patient | Christopher Baines | 2024-11-05 |
| * | Always assume that resource pool waiters will stick around•••As I think this is a more efficient design.
| Christopher Baines | 2024-11-05 |
| * | Move the inferior cleanup to the end of each system+target | Christopher Baines | 2024-11-04 |
| * | Refactor | Christopher Baines | 2024-11-03 |
| * | Use a separate db connection in channel-derivations-by-system-promise•••As this can run in parallel with something else using conn.
| Christopher Baines | 2024-11-02 |
| * | Increase the sleep-operation time for telling waiters to retry•••As this being too short was leading call-with-resource-pool to get stuck.
| Christopher Baines | 2024-10-31 |
| * | Finish chasing the call-with-resource-pool bug•••This took a while to find as process-job would just get stuck, and this wasn't
directly related to any particular change, just that more fibers increased the
chance of hitting it.
This commit includes lots of the things I changed while debugging.
| Christopher Baines | 2024-10-31 |
| * | Tweak the resource pool return behaviour•••If there's lots of contention for the resource pool, there will be lots of
waiters, so telling all of them to retry whenever a resource is returned seems
wasteful. This commit adds a new option (assume-reliable-waiters?) which will
have the resource pool try to give a returned resource to the oldest waiter,
if this fails, it'll go back to the old behaviour of telling all waiters to
retry.
| Christopher Baines | 2024-10-31 |
| * | Ensure that call-with-resource-from-pool doesn't get stuck•••As I think this was happening when it missed the resource-pool-retry-checkout
reply from the resource pool. Handle this case by periodically retrying with a
configurable timeout.
| Christopher Baines | 2024-10-31 |
| * | Add call-with-default-io-waiters•••As this is useful when starting threads from a fibers context.
| Christopher Baines | 2024-10-31 |
| * | Add new fibers utilities•••The new fibers-map uses the same batching approach that fibers-for-each uses,
and fibers-map-with-progress allows tracking on the results while the map is
happening.
| Christopher Baines | 2024-10-31 |
| * | Add make-queueing-channel•••From the build coordinator.
| Christopher Baines | 2024-10-31 |
| * | Add a comment | Christopher Baines | 2024-10-27 |
| * | Rework inserting derivations•••To add more parallelism.
| Christopher Baines | 2024-10-27 |
| * | Use a common derivation-ids-hash-table•••To allow for calling derivation-file-names->derivation-ids in parallel across
multiple fibers, using the PostgreSQL connection fiber to perform atomic
operations.
| Christopher Baines | 2024-10-27 |
| * | Add placeholder derivation source file nar procedures | Christopher Baines | 2024-10-27 |
| * | Include the package version in the "error when computing" messages•••As this helps when there are lots of packages with the same name.
| Christopher Baines | 2024-10-27 |
| * | Ensure the guix derivations are stored prior to attempting to substitute | Christopher Baines | 2024-10-27 |
| * | Add fibers-promise-reset | Christopher Baines | 2024-10-27 |