| Commit message (Expand) | Author | Age |
|---|
| * | Create index on build_starts•••As querying it is slow.
| Christopher Baines | 2023-11-24 |
| * | Include system uptime in the agent status information•••As I've found this useful in spotting systems which have problems.
| Christopher Baines | 2023-05-05 |
| * | Use an in memory table for the build_allocation_plan•••Since this doesn't need to be persisted and changes often.
| Christopher Baines | 2023-04-23 |
| * | Add processor count to the agent status•••This is useful when interpreting the load information.
| Christopher Baines | 2023-03-24 |
| * | Implement and extend the agent status functionality•••Previously, updating the status was used by the agent just to get back the
list of builds it was already allocated.
Now the status sent is actually stored, along with the 1min load average.
| Christopher Baines | 2023-03-22 |
| * | Store build and build results counts in the database•••Rather than trying to count all builds and build results at startup. This
should speed up the coordinator starting up, as currently it gets slower the
more builds and build results are in the database.
| Christopher Baines | 2022-10-28 |
| * | Record the details of derivation outputs•••I believe this will be useful when linking builds to other services, like the
Guix Data Service, since this information might be useful when joining up the
derivation being built to other derivations through the details of the outputs
produced.
| Christopher Baines | 2022-07-07 |
| * | Fix unprocessed_builds_with_derived_priorities•••A couple of the queries were wrong, this fixes it.
| Christopher Baines | 2021-12-22 |
| * | Add an index on output_metadata•••This should speed up fetching builds.
| Christopher Baines | 2021-11-18 |
| * | Add an active field to the agents table•••To allow making agents inactive to stop allocating builds to them.
| Christopher Baines | 2021-11-12 |
| * | Refactor how unprocessed builds are tracked•••Previously, the allocator worked out the derived priorities for each
build. Unfortunately this is quite a complex query, and took lots of time. As
a result of this, I think the WAL could grow excessively while this long query
was running.
To try and mitigate this, add a new table to keep track of the derived
priorities for unprocessed builds. This requires some maintenance to keep up
to date, which in turn will make things like submitting builds slower, but I
think this might help keep transaction length and WAL size down overall.
| Christopher Baines | 2021-11-07 |
| * | Introduce a systems table | Christopher Baines | 2021-05-21 |
| * | Introduce an outputs table | Christopher Baines | 2021-05-21 |
| * | Fix the foreign key constraint on setup_failure_missing_inputs•••It was broken in a previous migration.
| Christopher Baines | 2021-04-08 |
| * | Fix | Christopher Baines | 2021-03-30 |
| * | Fix types in the allocated_builds table | Christopher Baines | 2021-03-30 |
| * | Use numeric ids for the derivations and builds•••Using natural IDs was nice at the start, but just doesn't scale. This
migration cuts the database size, and potentially speeds up queries as well.
| Christopher Baines | 2021-03-29 |
| * | Add a new dynamic authentication approach•••This avoids the need to create agents upfront, which could be useful when
creating many childhurd VMs or using scheduling tools to dynamically run
agents.
| Christopher Baines | 2021-02-28 |
| * | Add tags for agents•••This will allow doing things like restricting builds by matching up there tags
to the tags of the agents.
| Christopher Baines | 2021-01-17 |
| * | Implement deferring builds•••This isn't intended as some time based scheduling, but more as a way to slow
down builds by deferring processing them until some point in the future.
I'm intending to use this to test fixed output derivations. I can look up all
the derivations I want to test, then defer the builds to run spread out across
some period. This feature saves having to submit the builds gradually.
| Christopher Baines | 2020-12-27 |
| * | Add an index on build_tags.build_id•••This makes fetching the tags for a build faster.
| Christopher Baines | 2020-12-21 |
| * | Implement build cancelation | Christopher Baines | 2020-12-16 |
| * | Add an unbuilt_outputs table•••One of the slow things in the derivation ordered allocator is working out what
outputs are unbuilt, as this requires looking at all the derivation
outputs (of which there are lots), and checking if any build exists which has
succeeded.
| Christopher Baines | 2020-11-06 |
| * | Store if the derivation is a fixed output derivation•••As this information will come in useful when working out how to handle builds
for fixed output derivations. Specifically, I want to make it configurable
whether to add builds for fixed output derivations if a build already exists
for the output, but the derivation is different.
Currently, different fixed output derivations can be ignored but it's not
possible to just avoid adding more builds for non fixed output derivations
while adding builds when fixed output derivations change. This new information
will help enable that.
| Christopher Baines | 2020-08-26 |
| * | Record what systems agents fetch builds for•••This can then be used by allocators to avoid allocating builds to agents that
they're never going to fetch.
| Christopher Baines | 2020-07-03 |
| * | Support tracking the end time of builds | Christopher Baines | 2020-07-01 |
| * | Support storing when builds start•••This isn't particularly accurate, what's actually being stored is the current
time when the record is inserted in to the coordinator database, but that
should happen just before the agent starts the build, so hopefully that's good
enough.
| Christopher Baines | 2020-07-01 |
| * | Support storing when builds are created | Christopher Baines | 2020-07-01 |
| * | Support adding tags to builds | Christopher Baines | 2020-05-31 |
| * | Add some more indexes to speed up derivation ordered allocation | Christopher Baines | 2020-05-11 |
| * | Replace datastore-fetch-input-builds-for-unprocessed-builds•••It worked under some database conditions, but was very slow under others. Move
more of the logic in to SQL in an attempt to make the allocator faster. This
sort of works, but there were some advantages to the approach before the
approach being replaced in this commit.
| Christopher Baines | 2020-05-10 |
| * | Add datastore-fetch-unprocessed-builds-with-propagated-priorities•••To use with the derivation ordered allocator.
| Christopher Baines | 2020-05-10 |
| * | Add sqlite datastore support for storing unprocessed hook events | Christopher Baines | 2020-05-08 |
| * | Add a couple of indexes that should speed up allocating builds | Christopher Baines | 2020-04-29 |
| * | Create an index on derivation_outputs.output•••This is important to speed up looking for derivations that provide an output,
that's used in the allocation process.
| Christopher Baines | 2020-04-28 |
| * | Send over some metadata from the agent for each output•••This will hopefully make it easier to create narinfo files for the outputs. I
think all of this information can be derived from the nar, but I'm not sure
how to do that, so maybe this can eventually be removed.
| Christopher Baines | 2020-04-23 |
| * | Add a new table to store setup_failures•••This is when a build was allocated to an agent, but the agent couldn't setup
the environment for the build. One failure I'm particularly thinking about is
where inputs to the derivation are missing, so add another table to store
them.
| Christopher Baines | 2020-04-13 |
| * | Add the build_results table | Christopher Baines | 2020-04-13 |
| * | Create tables for allocating builds•••One table to store which build is allocated to which agent, and another to
store a "plan" of allocations. For this plan, a build can be potentially
allocated to multiple agents, and which agent it will be allocated to depends
on which agent claims it first.
| Christopher Baines | 2020-04-10 |
| * | Add initial agent related tables | Christopher Baines | 2020-04-10 |
| * | Add a migration to create the builds table | Christopher Baines | 2020-04-10 |
| * | Add tables for derivations | Christopher Baines | 2020-04-04 |
| * | Add an initial migration | Christopher Baines | 2020-04-03 |
| * | Add sqitch conf and empty plan | Christopher Baines | 2020-04-03 |