| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Switch to using = for testing numerical equality | Christopher Baines | 2022-10-22 |
| | | | | | As I think I've been abusing eq?. | ||
| * | Clean up some logging | Christopher Baines | 2022-10-22 |
| | | |||
| * | Log when output files are renamed | Christopher Baines | 2022-10-22 |
| | | |||
| * | Add more logging when output files are deleted | Christopher Baines | 2022-10-22 |
| | | |||
| * | Tweak upload handling again | Christopher Baines | 2022-10-21 |
| | | | | | | | This partially reverts some recent changes. There should now be better handling of when all the bytes have been sent, but the hash hasn't yet been computed. | ||
| * | Copy upload error handling to the partial route | Christopher Baines | 2022-10-21 |
| | | |||
| * | Log the file size and md5 hash on the agent side | Christopher Baines | 2022-10-21 |
| | | | | | When errors occur during upload. | ||
| * | Include the file size and md5 hash in error messages | Christopher Baines | 2022-10-21 |
| | | | | | | When an error occurs while trying to compute the hash, as I hope this information will help to identify where things have gone wrong. | ||
| * | Tweak output uploading code | Christopher Baines | 2022-10-19 |
| | | | | | Force sending the file from the start the first time the upload is attempted. | ||
| * | Use suspendable ports for the agent, with timeouts | Christopher Baines | 2022-10-16 |
| | | | | | | | | | This seems like a way of making the Guile internals for doing network I/O reliable. Currently, there are problems where things on the network timeout, but the Guile code for reading/writing just sits there, hung. This seems like it might help. | ||
| * | Throttle calling alarm | Christopher Baines | 2022-10-15 |
| | | | | | Just in case calling it frequently causes problems. | ||
| * | Reset the upload progress when appropriate | Christopher Baines | 2022-10-14 |
| | | |||
| * | Add more logging around file uploads | Christopher Baines | 2022-10-14 |
| | | |||
| * | Shorten timeout when submitting outputs | Christopher Baines | 2022-10-14 |
| | | | | | By resetting the timeout while the file is being uploaded. | ||
| * | Tweak output upload timeout | Christopher Baines | 2022-10-14 |
| | | | | | | Make it longer to avoid timeouts for large files, and increase the delay before restarting the upload. | ||
| * | Don't crash on chunked input exceptions | Christopher Baines | 2022-10-08 |
| | | |||
| * | Check the dynamic auth token before creating new agents | Christopher Baines | 2022-10-08 |
| | | |||
| * | Move retry in submit-output | Christopher Baines | 2022-04-09 |
| | | |||
| * | Add a timeout for submitting outputs | Christopher Baines | 2022-04-09 |
| | | |||
| * | Only use GC protection when gnutls won't internally retry | Christopher Baines | 2022-02-04 |
| | | |||
| * | Fix the threshold for metrics delay logging | Christopher Baines | 2022-02-01 |
| | | |||
| * | Tweak metrics delay logging | Christopher Baines | 2022-01-20 |
| | | | | | | Just instrument the update-managed-metrics! function, and move some code around so this is clearer in the logs. | ||
| * | Remove bodies from responses to HEAD requests | Christopher Baines | 2021-12-27 |
| | | |||
| * | Fix route for getting the bytes uploaded for an output | Christopher Baines | 2021-12-24 |
| | | |||
| * | Improve still more to send log message | Christopher Baines | 2021-12-22 |
| | | |||
| * | Improve logging when submitting outputs | Christopher Baines | 2021-12-22 |
| | | |||
| * | Check before deleting files | Christopher Baines | 2021-11-26 |
| | | | | | As I've seen exceptions here. | ||
| * | Remove redundant if in the controller | Christopher Baines | 2021-11-26 |
| | | |||
| * | Don't print backtraces in the controller when chunked inputs end | Christopher Baines | 2021-11-26 |
| | | |||
| * | Delete existing files when processing upload requests | Christopher Baines | 2021-11-22 |
| | | | | | I think this will help when handling new requests after failed ones. | ||
| * | Unwind on some exceptions | Christopher Baines | 2021-11-22 |
| | | | | | The error handling here should be handling by unwinding. | ||
| * | Improve some way numbers are displayed | Christopher Baines | 2021-11-22 |
| | | |||
| * | Only check the size of the file once when uploading | Christopher Baines | 2021-11-21 |
| | | |||
| * | Fix variable reference in submit-output | Christopher Baines | 2021-11-21 |
| | | |||
| * | Compress outputs outside of the upload slot | Christopher Baines | 2021-11-20 |
| | | | | | | So that the only thing taking place in the upload slot, is the actual upload, which should improve throughput. | ||
| * | Track delays for reporting metrics | Christopher Baines | 2021-11-16 |
| | | |||
| * | Check if an output has been uploaded before trying to upload it | Christopher Baines | 2021-11-16 |
| | | | | | | | | | | | This can help if the output has been uploaded, but the hash isn't present, since trying to submit the build result will prompt for the output to be sent again, but it doesn't need to be, the agent just needs to wait. This is a little inelegant, maybe there needs to be some way for the agent to explicitly check for the hash to be computed, but I'm hoping these changes will help with uploading large outputs. | ||
| * | Add error handling around computing output hashes | Christopher Baines | 2021-11-15 |
| | | | | | As I've seen decompression errors. | ||
| * | Fix the uri when calling coordinator-handle-failed-request | Christopher Baines | 2021-11-15 |
| | | |||
| * | Handle the case where there are no more bytes to send | Christopher Baines | 2021-11-15 |
| | | | | | | | | | | When submitting an output. This also fixes a regression in not passing report-bytes-sent on to call-with-streaming-http-request. I think this case where the agent is trying to send 0 bytes to the coordinator can happen when the last request to the coordinator times out, probably due to the computing of the hash taking so long. | ||
| * | Remove some test code | Christopher Baines | 2021-11-14 |
| | | |||
| * | Implement initial support for resuming HTTP uploads | Christopher Baines | 2021-11-14 |
| | | | | | | This means agents reattempting uploads don't have to start from scratch, and can instead pick up from what's already been uploaded to the coordinator. | ||
| * | Don't error for 404 responses in coordinator-http-request | Christopher Baines | 2021-11-14 |
| | | |||
| * | Don't error for responses with no body in coordinator-http-request | Christopher Baines | 2021-11-14 |
| | | |||
| * | Start checking the hashes of submitted outputs | Christopher Baines | 2021-11-14 |
| | | | | | | | | | | | This provides some extra safety on top of the guarantees from TCP around the integrity of the data received. I'm introducing this now in preparation for supporting resuming partial uploads. Because this will add some extra complexity around receiving uploads, this extra check should ensure that issues with the implementation cannot lead to corrupt uploads. | ||
| * | Return to compressing outputs then sending them | Christopher Baines | 2021-11-07 |
| | | | | | | | | | | | | | | Trying to avoid the GnuTLS bindings breaking when the garbage collector runs is quite difficult, and the current approach isn't very effective. I want to try instead to support resuming partial uploads, as that should both help with the GnuTLS GC issue, as well as network interruptions in general. I think that approach is going to be easier with compressing the files up-front, so revert to doing that. This partially reverts commit 8258e9c8d9f729b2670a602c523c59847b676b1a. | ||
| * | Move retrying uploads out of the with-upload-slot region | Christopher Baines | 2021-08-07 |
| | | | | | | Such that the retry happens with a fresh slot (and the associated tracking information). | ||
| * | Support reporting bytes sent when submitting outputs | Christopher Baines | 2021-06-08 |
| | | |||
| * | Retry more when sending outputs | Christopher Baines | 2021-05-30 |
| | | | | | | Since time has been spent building them, so wait longer before giving up submitting the outputs. | ||
| * | Improve the reveived output message | Christopher Baines | 2021-05-30 |
| | | |||