| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Create an index on derivation_outputs.output | Christopher Baines | 2020-04-28 |
| | | | | | | This is important to speed up looking for derivations that provide an output, that's used in the allocation process. | ||
| * | Fix metric namespacing in a few places | Christopher Baines | 2020-04-28 |
| | | |||
| * | Track the duration of the build allocation | Christopher Baines | 2020-04-28 |
| | | |||
| * | Start tracking the duration of a few datastore functions | Christopher Baines | 2020-04-28 |
| | | |||
| * | Add metrics tools for histograms | Christopher Baines | 2020-04-28 |
| | | |||
| * | Use max, rather than min for the parallelism | Christopher Baines | 2020-04-28 |
| | | | | | 4 should be the lower bound, not the upper bound! | ||
| * | Rework the way metrics are handled | Christopher Baines | 2020-04-28 |
| | | | | | | Start writing a proper Prometheus client, hopefully this code can be extracted at some point. | ||
| * | Increase the planned builds for agents | Christopher Baines | 2020-04-28 |
| | | | | | | As the allocator is so slow at the moment, that agents are running out of work. | ||
| * | Add datastore-find-build-result | Christopher Baines | 2020-04-27 |
| | | |||
| * | Support listing processed builds | Christopher Baines | 2020-04-27 |
| | | |||
| * | Guard against a build being processed twice | Christopher Baines | 2020-04-27 |
| | | | | | | | | Or rather reported as finished twice, which could happen if the agent retries the request, but it worked the first time. The database constraint will prevent the request going through, but check here to provide a better error message. | ||
| * | Return a boolean value for processed | Christopher Baines | 2020-04-27 |
| | | |||
| * | Retry requests a few times if they error | Christopher Baines | 2020-04-27 |
| | | | | | This should make agent <-> coordinator communication more failure tolerant. | ||
| * | Add a function to retry a thunk a number of times | Christopher Baines | 2020-04-27 |
| | | |||
| * | Add more error handling to coordinator-http-request | Christopher Baines | 2020-04-27 |
| | | |||
| * | Say what log file is being uploaded | Christopher Baines | 2020-04-27 |
| | | |||
| * | Refactor some of the HTTP requests | Christopher Baines | 2020-04-27 |
| | | | | | To reduce the code duplication. | ||
| * | Improve the output from the agent process | Christopher Baines | 2020-04-27 |
| | | |||
| * | Impose a limit on the number of planned builds | Christopher Baines | 2020-04-27 |
| | | | | | This helps to avoid planning too far ahead. This should be configurable. | ||
| * | Update the comment regarding processing chunked requests | Christopher Baines | 2020-04-27 |
| | | | | | The reasoning changed with 9acc42cfd1d081ca1e8d8aa41fe0eb20303e5df3. | ||
| * | Revert "Revert "Read chunked request bodies in different threads to avoid ↵ | Christopher Baines | 2020-04-27 |
| | | | | | | | | | | blocking"" Fibers gets upset when trying to upload files: Attempt to suspend fiber within continuation barrier, so try doing this in a thread. This reverts commit 63b0fa9c37a5cd540bcbb975598baa8ef1a3831b. | ||
| * | Revert "Switch to the non-fibers web server" | Christopher Baines | 2020-04-27 |
| | | | | | | | The non-fibers one processes requests sequentially, which just won't do. This reverts commit 2d2ae5b6b01081f35ad2fc25affdef8c876b48e6. | ||
| * | Revert "Read chunked request bodies in different threads to avoid blocking" | Christopher Baines | 2020-04-27 |
| | | | | | | | | This didn't help, as I think the web server just processes requests sequentially. Hopefully once that's resolved, this won't be necessary. This reverts commit 48ed0eead59119c269993dbfa2c9ca8302b52d3d. | ||
| * | Don't expect substitute-derivation to evaluate to #t | Christopher Baines | 2020-04-26 |
| | | | | | It raises an exception if unsuccessful now. | ||
| * | Improve error handling in substitute-derivation | Christopher Baines | 2020-04-26 |
| | | |||
| * | Ensure that successful builds have the outputs and log file | Christopher Baines | 2020-04-26 |
| | | | | | Present on the coordinator. | ||
| * | Change datastore-list-build-outputs to work without metadata | Christopher Baines | 2020-04-26 |
| | | |||
| * | Read chunked request bodies in different threads to avoid blocking | Christopher Baines | 2020-04-26 |
| | | | | | | Not sure if this'll work, but hopefully offloading reading the chunked requests to other threads will avoid blocking. | ||
| * | Try to fix handling of chuncked requests | Christopher Baines | 2020-04-26 |
| | | | | | | | | | | | | | | From looking at what curl does, it seems that the last the requests end in "0\r\n\r\n". The requests being sent before just had "0\r\n" at the end. This worked with the server, because that wasn't expecting the final "\r\n", and it would crash if it was included, as it would be read as the start of the next request. To work around this, adjust both the sending and receiving of the requests. Send the "\r\n" after the chuncked data when making requests, and use a patched version of make-chunked-input-port that requests two more bytes after it's finished reading the last chunk. | ||
| * | Add Content-Type to the requests for file uploads | Christopher Baines | 2020-04-26 |
| | | | | | This is probably sensible. | ||
| * | Expose metrics | Christopher Baines | 2020-04-26 |
| | | |||
| * | Add a new metrics module | Christopher Baines | 2020-04-26 |
| | | | | | To produce Prometheus style metrics for the counts of various things. | ||
| * | Add datastore functions to get metrics | Christopher Baines | 2020-04-26 |
| | | |||
| * | Make it possible to pass in substitute-urls to the agent | Christopher Baines | 2020-04-26 |
| | | | | | So that you don't have to just use the daemon's defaults. | ||
| * | Respect the allocation plan ordering when providing builds to agents | Christopher Baines | 2020-04-25 |
| | | |||
| * | Start implementing prioritisation of builds | Christopher Baines | 2020-04-25 |
| | | |||
| * | Fix datastore-find-build-output | Christopher Baines | 2020-04-25 |
| | | | | | | I don't think when was really working, the procedure was returning <undefined> when it shouldn't have been. | ||
| * | Handle unknown build results in the allocator | Christopher Baines | 2020-04-25 |
| | | |||
| * | Left join on build_results in datastore-list-builds-for-output | Christopher Baines | 2020-04-25 |
| | | | | | So that it includes builds that haven't been processed yet in the results. | ||
| * | Add a script to fetch builds from the Guix Data Service | Christopher Baines | 2020-04-25 |
| | | | | | As that is an easy way to find things to build. | ||
| * | Fix issues with fetching builds for agents | Christopher Baines | 2020-04-25 |
| | | | | | | | | Because the allocation plan can be replaced with one that's already out of date, including allocating builds that have been completed, guard against this here. This is a comprimise to avoid having to block operations when planning allocating builds. | ||
| * | Add helpful output to the agent process | Christopher Baines | 2020-04-25 |
| | | |||
| * | Don't output the build log on the agent process | Christopher Baines | 2020-04-25 |
| | | | | | | This isn't particularly helpful, especially as the agent process now handles the log file. | ||
| * | Support sending and storing log files for builds | Christopher Baines | 2020-04-25 |
| | | |||
| * | Extract out call-with-streaming-http-request | Christopher Baines | 2020-04-25 |
| | | |||
| * | Restructure the post build actions | Christopher Baines | 2020-04-25 |
| | | |||
| * | Add options suited for providing substitutes | Christopher Baines | 2020-04-25 |
| | | | | | | | | | | | Allow specifying build priority, although the allocator currently doesn't use this. Add --defer-allocation to allow inserting lots of builds without spending time re-computing the allocation for each one. Add --ensure-all-related-derivations-have-builds to make it easy to have a derivation, and all related derivations built at least once. Add --ignore-if-build-for-derivation-exists to make it easy to avoid building derivations again if that isn't the intention. | ||
| * | Add datastore-list-related-derivations-with-no-build | Christopher Baines | 2020-04-25 |
| | | | | | | Useful for finding out what derivations need to be built to ensure all related derivations have been built by the build coordinator. | ||
| * | Add datastore-list-builds-for-derivation | Christopher Baines | 2020-04-25 |
| | | |||
| * | Add a hook to generate the files needed to provide substitutes | Christopher Baines | 2020-04-23 |
| | | |||