aboutsummaryrefslogtreecommitdiff
path: root/sqitch
Commit message (Collapse)AuthorAge
* Store build and build results counts in the databaseChristopher Baines2022-10-28
| | | | | | Rather than trying to count all builds and build results at startup. This should speed up the coordinator starting up, as currently it gets slower the more builds and build results are in the database.
* Record the details of derivation outputsChristopher Baines2022-07-07
| | | | | | | I believe this will be useful when linking builds to other services, like the Guix Data Service, since this information might be useful when joining up the derivation being built to other derivations through the details of the outputs produced.
* Fix unprocessed_builds_with_derived_prioritiesChristopher Baines2021-12-22
| | | | A couple of the queries were wrong, this fixes it.
* Add an index on output_metadataChristopher Baines2021-11-18
| | | | This should speed up fetching builds.
* Add an active field to the agents tableChristopher Baines2021-11-12
| | | | To allow making agents inactive to stop allocating builds to them.
* Refactor how unprocessed builds are trackedChristopher Baines2021-11-07
| | | | | | | | | | | | Previously, the allocator worked out the derived priorities for each build. Unfortunately this is quite a complex query, and took lots of time. As a result of this, I think the WAL could grow excessively while this long query was running. To try and mitigate this, add a new table to keep track of the derived priorities for unprocessed builds. This requires some maintenance to keep up to date, which in turn will make things like submitting builds slower, but I think this might help keep transaction length and WAL size down overall.
* Fix part of the create_outputs migrationChristopher Baines2021-05-27
| | | | | It was completely broken, since it would just delete all entries from the unbuilt outputs table.
* Introduce a systems tableChristopher Baines2021-05-21
|
* Introduce an outputs tableChristopher Baines2021-05-21
|
* Fix the foreign key constraint on setup_failure_missing_inputsChristopher Baines2021-04-08
| | | | It was broken in a previous migration.
* FixChristopher Baines2021-03-30
|
* Fix types in the allocated_builds tableChristopher Baines2021-03-30
|
* Use numeric ids for the derivations and buildsChristopher Baines2021-03-29
| | | | | Using natural IDs was nice at the start, but just doesn't scale. This migration cuts the database size, and potentially speeds up queries as well.
* Add a new dynamic authentication approachChristopher Baines2021-02-28
| | | | | | This avoids the need to create agents upfront, which could be useful when creating many childhurd VMs or using scheduling tools to dynamically run agents.
* Add tags for agentsChristopher Baines2021-01-17
| | | | | This will allow doing things like restricting builds by matching up there tags to the tags of the agents.
* Implement deferring buildsChristopher Baines2020-12-27
| | | | | | | | | This isn't intended as some time based scheduling, but more as a way to slow down builds by deferring processing them until some point in the future. I'm intending to use this to test fixed output derivations. I can look up all the derivations I want to test, then defer the builds to run spread out across some period. This feature saves having to submit the builds gradually.
* Add an index on build_tags.build_idChristopher Baines2020-12-21
| | | | This makes fetching the tags for a build faster.
* Implement build cancelationChristopher Baines2020-12-16
|
* Add an unbuilt_outputs tableChristopher Baines2020-11-06
| | | | | | | One of the slow things in the derivation ordered allocator is working out what outputs are unbuilt, as this requires looking at all the derivation outputs (of which there are lots), and checking if any build exists which has succeeded.
* Store if the derivation is a fixed output derivationChristopher Baines2020-08-26
| | | | | | | | | | | | As this information will come in useful when working out how to handle builds for fixed output derivations. Specifically, I want to make it configurable whether to add builds for fixed output derivations if a build already exists for the output, but the derivation is different. Currently, different fixed output derivations can be ignored but it's not possible to just avoid adding more builds for non fixed output derivations while adding builds when fixed output derivations change. This new information will help enable that.
* Record what systems agents fetch builds forChristopher Baines2020-07-03
| | | | | This can then be used by allocators to avoid allocating builds to agents that they're never going to fetch.
* Support tracking the end time of buildsChristopher Baines2020-07-01
|
* Support storing when builds startChristopher Baines2020-07-01
| | | | | | | This isn't particularly accurate, what's actually being stored is the current time when the record is inserted in to the coordinator database, but that should happen just before the agent starts the build, so hopefully that's good enough.
* Support storing when builds are createdChristopher Baines2020-07-01
|
* Support adding tags to buildsChristopher Baines2020-05-31
|
* Add some more indexes to speed up derivation ordered allocationChristopher Baines2020-05-11
|
* Replace datastore-fetch-input-builds-for-unprocessed-buildsChristopher Baines2020-05-10
| | | | | | | It worked under some database conditions, but was very slow under others. Move more of the logic in to SQL in an attempt to make the allocator faster. This sort of works, but there were some advantages to the approach before the approach being replaced in this commit.
* Add datastore-fetch-unprocessed-builds-with-propagated-prioritiesChristopher Baines2020-05-10
| | | | To use with the derivation ordered allocator.
* Add sqlite datastore support for storing unprocessed hook eventsChristopher Baines2020-05-08
|
* Add a couple of indexes that should speed up allocating buildsChristopher Baines2020-04-29
|
* Create an index on derivation_outputs.outputChristopher Baines2020-04-28
| | | | | This is important to speed up looking for derivations that provide an output, that's used in the allocation process.
* Send over some metadata from the agent for each outputChristopher Baines2020-04-23
| | | | | | This will hopefully make it easier to create narinfo files for the outputs. I think all of this information can be derived from the nar, but I'm not sure how to do that, so maybe this can eventually be removed.
* Add a new table to store setup_failuresChristopher Baines2020-04-13
| | | | | | | This is when a build was allocated to an agent, but the agent couldn't setup the environment for the build. One failure I'm particularly thinking about is where inputs to the derivation are missing, so add another table to store them.
* Add the build_results tableChristopher Baines2020-04-13
|
* Create tables for allocating buildsChristopher Baines2020-04-10
| | | | | | | One table to store which build is allocated to which agent, and another to store a "plan" of allocations. For this plan, a build can be potentially allocated to multiple agents, and which agent it will be allocated to depends on which agent claims it first.
* Add initial agent related tablesChristopher Baines2020-04-10
|
* Add a migration to create the builds tableChristopher Baines2020-04-10
|
* Add tables for derivationsChristopher Baines2020-04-04
|
* Add an initial migrationChristopher Baines2020-04-03
|
* Add sqitch conf and empty planChristopher Baines2020-04-03
generated by cgit v1.2.3 (git 2.45.0) at 2024-11-28 06:23:21 +0000