aboutsummaryrefslogtreecommitdiff
path: root/guix-build-coordinator
Commit message (Collapse)AuthorAge
* Try to fix handling of chuncked requestsChristopher Baines2020-04-26
| | | | | | | | | | | | | From looking at what curl does, it seems that the last the requests end in "0\r\n\r\n". The requests being sent before just had "0\r\n" at the end. This worked with the server, because that wasn't expecting the final "\r\n", and it would crash if it was included, as it would be read as the start of the next request. To work around this, adjust both the sending and receiving of the requests. Send the "\r\n" after the chuncked data when making requests, and use a patched version of make-chunked-input-port that requests two more bytes after it's finished reading the last chunk.
* Add Content-Type to the requests for file uploadsChristopher Baines2020-04-26
| | | | This is probably sensible.
* Expose metricsChristopher Baines2020-04-26
|
* Add a new metrics moduleChristopher Baines2020-04-26
| | | | To produce Prometheus style metrics for the counts of various things.
* Add datastore functions to get metricsChristopher Baines2020-04-26
|
* Make it possible to pass in substitute-urls to the agentChristopher Baines2020-04-26
| | | | So that you don't have to just use the daemon's defaults.
* Respect the allocation plan ordering when providing builds to agentsChristopher Baines2020-04-25
|
* Start implementing prioritisation of buildsChristopher Baines2020-04-25
|
* Fix datastore-find-build-outputChristopher Baines2020-04-25
| | | | | I don't think when was really working, the procedure was returning <undefined> when it shouldn't have been.
* Handle unknown build results in the allocatorChristopher Baines2020-04-25
|
* Left join on build_results in datastore-list-builds-for-outputChristopher Baines2020-04-25
| | | | So that it includes builds that haven't been processed yet in the results.
* Fix issues with fetching builds for agentsChristopher Baines2020-04-25
| | | | | | | Because the allocation plan can be replaced with one that's already out of date, including allocating builds that have been completed, guard against this here. This is a comprimise to avoid having to block operations when planning allocating builds.
* Add helpful output to the agent processChristopher Baines2020-04-25
|
* Don't output the build log on the agent processChristopher Baines2020-04-25
| | | | | This isn't particularly helpful, especially as the agent process now handles the log file.
* Support sending and storing log files for buildsChristopher Baines2020-04-25
|
* Extract out call-with-streaming-http-requestChristopher Baines2020-04-25
|
* Restructure the post build actionsChristopher Baines2020-04-25
|
* Add options suited for providing substitutesChristopher Baines2020-04-25
| | | | | | | | | | Allow specifying build priority, although the allocator currently doesn't use this. Add --defer-allocation to allow inserting lots of builds without spending time re-computing the allocation for each one. Add --ensure-all-related-derivations-have-builds to make it easy to have a derivation, and all related derivations built at least once. Add --ignore-if-build-for-derivation-exists to make it easy to avoid building derivations again if that isn't the intention.
* Add datastore-list-related-derivations-with-no-buildChristopher Baines2020-04-25
| | | | | Useful for finding out what derivations need to be built to ensure all related derivations have been built by the build coordinator.
* Add datastore-list-builds-for-derivationChristopher Baines2020-04-25
|
* Add a hook to generate the files needed to provide substitutesChristopher Baines2020-04-23
|
* Add datastore-list-build-outputsChristopher Baines2020-04-23
|
* Add datastore-find-derivation-systemChristopher Baines2020-04-23
|
* Add a function to generate the string for a narinfo fileChristopher Baines2020-04-23
|
* Send over some metadata from the agent for each outputChristopher Baines2020-04-23
| | | | | | This will hopefully make it easier to create narinfo files for the outputs. I think all of this information can be derived from the nar, but I'm not sure how to do that, so maybe this can eventually be removed.
* Add a hook to handle missing inputsChristopher Baines2020-04-17
| | | | | | | That submits new build jobs to build these missing inputs if appropriate. This means that you can tell the coordinator to build something, and it will automatically attempt to build the dependencies if they're missing.
* Switch the build-allocator thread to not use a channelChristopher Baines2020-04-17
| | | | | The put-message operation blocks, which doesn't work for triggering the allocation process.
* Only sleep in the agent when there are no buildsChristopher Baines2020-04-17
| | | | The last time the agent checked.
* Switch to the fibers backend for the webserverChristopher Baines2020-04-17
| | | | This might help with using fibers for other things.
* Re-work how the datastore module re-exports thingsChristopher Baines2020-04-17
| | | | | This seems to avoid all the warnings, and fix the broken merge-generics behaviour.
* Add some more sqlite datastore functionsChristopher Baines2020-04-17
|
* Wrap transactions with some error handlingChristopher Baines2020-04-17
| | | | To rollback when they fail.
* Copy a few SQlite options from CuirassChristopher Baines2020-04-17
| | | | They'll probably help.
* Implement a couple of basic hooksChristopher Baines2020-04-16
| | | | This allows configurable code to be executed when builds succeed or fail.
* Add better error handling in to the agentChristopher Baines2020-04-16
| | | | So that it reports issues to the coordinator, rather than just crashing.
* Include finished builds in the agent for build queryChristopher Baines2020-04-16
|
* Improve build allocationChristopher Baines2020-04-13
| | | | | | | | | Move some of the code around, and trigger allocating builds via a thread if an agent fails to setup for a build and when a build succeeds/fails. This is important, as some setup failures can be handled by the build allocator, for example a build finishing may unblock other builds waiting for outputs it generates.
* Support emptying the build planChristopher Baines2020-04-13
| | | | Previously, it would error trying to insert 0 records.
* Add missing importChristopher Baines2020-04-13
|
* Don't allocate builds to agents where the setup previously failedChristopher Baines2020-04-13
|
* Add datastore-fetch-setup-failuresChristopher Baines2020-04-13
|
* Fix printing out requests to the coordinatorChristopher Baines2020-04-13
|
* Support reporting setup failures to the coordinatorChristopher Baines2020-04-13
|
* Add datastore methods for handling setup failuresChristopher Baines2020-04-13
|
* Have agents run in a loopChristopher Baines2020-04-13
| | | | Perform a build, then query for the next one.
* Only download missing inputs if all have substitutesChristopher Baines2020-04-13
| | | | | | | To avoid potentially wasting time. Instead, report the missing inputs to the coordinator as soon as possible. The build may be scheduled on a different agent, so it might not be necessary to download the inputs which do have substitutes available.
* Add support for sending/receiving build resultsChristopher Baines2020-04-13
|
* Add datastore-store-build-resultChristopher Baines2020-04-13
|
* Re-work how agents handle missing inputsChristopher Baines2020-04-13
| | | | Don't start the build if there are missing inputs.
* Use substitute-derivation in the coordinatorChristopher Baines2020-04-13
| | | | So that it can read the derivation, and store the details in the database.