aboutsummaryrefslogtreecommitdiff
path: root/guix-build-coordinator/build-allocator.scm
Commit message (Collapse)AuthorAge
* Rework how the derivation ordered allocator gets buildsChristopher Baines2020-11-06
| | | | | | Use a temporary table to avoid computing the priorities for all builds. This speeds up the allocation to only take a few seconds on the database I'm testing against.
* Attempt to more gracefully handle the problem of missing derivationsChristopher Baines2020-11-02
| | | | In the agent and allocator.
* Only consider builds created in the last two weeksChristopher Baines2020-10-29
| | | | | | For the derivation ordered allocator. This is an quick alternative for having some kind of archival mechanism for builds. It should reduce the time it takes the allocator to run.
* Track the number of builds the allocator is consideringChristopher Baines2020-09-23
|
* Factor in which systems agents fetch in the allocation processChristopher Baines2020-07-03
|
* Pass through the db connection in datastore-call-with-transactionChristopher Baines2020-06-20
|
* Fetch the derivation ordered allocator data in a transactionChristopher Baines2020-06-20
| | | | To try and avoid the data from the two queries being inconsistent.
* Fetch the unprocessed builds after the priorititesChristopher Baines2020-05-20
| | | | | This might help avoid any issue with having a unprocessed build, but not knowing the priority, as it's just been processed.
* Remove some unused codeChristopher Baines2020-05-17
|
* Speed up the derivation ordered allocator a little bitChristopher Baines2020-05-11
| | | | | Use EXCEPT, rather than NOT IN to make the SQL query faster. Also, just return and use the build id, rather than a alist.
* Replace datastore-fetch-input-builds-for-unprocessed-buildsChristopher Baines2020-05-10
| | | | | | | It worked under some database conditions, but was very slow under others. Move more of the logic in to SQL in an attempt to make the allocator faster. This sort of works, but there were some advantages to the approach before the approach being replaced in this commit.
* Use some more SQL to speed up the derivation ordered allocatorChristopher Baines2020-05-10
|
* Fix datastore-fetch-input-builds-for-unprocessed-buildsChristopher Baines2020-05-10
| | | | | Previously it was ignoring outputs without builds or build results. This fixes that.
* Speed up part of the derivation ordered allocatorChristopher Baines2020-05-10
| | | | By using a bigger SQL query, rather than lots of small ones.
* Stop the derivation-ordered allocator using missing inputsChristopher Baines2020-05-10
| | | | Because it's building everything, missing inputs shouldn't be as relevant.
* Add a new allocation strategyChristopher Baines2020-05-08
| | | | | This should work better where the build coordinator is being used to build all derivations in the graph.
* Fix the removal of missing input setup failures in the allocatorChristopher Baines2020-05-01
| | | | | Previously it was ignoring all setup failures when a single one of the outputs for a single setup failure should be available.
* Try to better order the buildsChristopher Baines2020-05-01
| | | | | | As well as sorting by priority, this now looks at which builds are missing outputs, and which builds will provide those outputs and tries to order the builds providing the outputs before the builds that need them.
* Have the build priorities propagate between buildsChristopher Baines2020-04-29
| | | | | | | | | This should at least do a better job of boosting the priority of builds required to perform builds which have high priorities. I'm expecting this to result in contiguous chunks of builds with all the same priority, and the ordering within those chunks isn't yet decided intelligently.
* Move output-has-successful-build? out from filter-builds-for-agentChristopher Baines2020-04-29
| | | | As it's not dependent on the agent.
* Memoize output-has-successful-build?Christopher Baines2020-04-29
| | | | This is called for the same outputs multiple times, so memoize the result.
* Add a useful commentChristopher Baines2020-04-29
|
* Use a hash-table for setup failures, rather than an alistChristopher Baines2020-04-29
| | | | This will probably perform better when there are lots of step failures.
* Increase the planned builds for agentsChristopher Baines2020-04-28
| | | | | As the allocator is so slow at the moment, that agents are running out of work.
* Impose a limit on the number of planned buildsChristopher Baines2020-04-27
| | | | This helps to avoid planning too far ahead. This should be configurable.
* Start implementing prioritisation of buildsChristopher Baines2020-04-25
|
* Handle unknown build results in the allocatorChristopher Baines2020-04-25
|
* Add a hook to handle missing inputsChristopher Baines2020-04-17
| | | | | | | That submits new build jobs to build these missing inputs if appropriate. This means that you can tell the coordinator to build something, and it will automatically attempt to build the dependencies if they're missing.
* Improve build allocationChristopher Baines2020-04-13
Move some of the code around, and trigger allocating builds via a thread if an agent fails to setup for a build and when a build succeeds/fails. This is important, as some setup failures can be handled by the build allocator, for example a build finishing may unblock other builds waiting for outputs it generates.