| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Actually use non-blocking ports for network requests | Christopher Baines | 2024-03-02 |
| | | | | | In most places at least. | ||
| * | Change retry-on-error to take #:ignore and #:no-retry | Christopher Baines | 2024-01-12 |
| | | | | | And change #:ignore to better reflect ignoring the exception. | ||
| * | Don't call report-bytes-hashed with #f | Christopher Baines | 2023-05-24 |
| | | | | | Just log the line instead. | ||
| * | Show backtraces for perform-upload errors | Christopher Baines | 2023-05-23 |
| | | |||
| * | Tweak logging around agent output submission | Christopher Baines | 2023-05-22 |
| | | |||
| * | Fix retry-times not always being set | Christopher Baines | 2023-05-11 |
| | | |||
| * | Clean up some handling of uploads for agents | Christopher Baines | 2023-05-11 |
| | | | | | This commit should correct the progress reporting on partial uploads. | ||
| * | Have agents report on the progress of the coordinator hashing outputs | Christopher Baines | 2023-05-11 |
| | | | | | Otherwise it looks like the upload should finish, but hasn't. | ||
| * | Tweak retrying for status update requests | Christopher Baines | 2023-05-10 |
| | | | | | | Don't retry status updates many times, since the information will be more out of date each time. | ||
| * | Change submit-output to not spend so much time waiting | Christopher Baines | 2023-05-08 |
| | | | | | | Make use of the coordinator trying to avoid the connection timing out. This should improve things when the coordinator is restarted or crashes. | ||
| * | Stop using chunked transfers for file uploads | Christopher Baines | 2023-05-08 |
| | | | | | | As the amount of data to upload is known, this is unnecessary complexity and overhead. | ||
| * | Add some comments about streaming http requests | Christopher Baines | 2023-05-05 |
| | | |||
| * | Include system uptime in the agent status information | Christopher Baines | 2023-05-05 |
| | | | | | As I've found this useful in spotting systems which have problems. | ||
| * | Remove now redundant logging around gc protection | Christopher Baines | 2023-05-03 |
| | | |||
| * | Stop monitoring uploads through the chunked output port | Christopher Baines | 2023-04-30 |
| | | | | | Use the dump-port* progress reporter instead. | ||
| * | Deallocate canceled builds from agents when they startup | Christopher Baines | 2023-04-21 |
| | | |||
| * | Remove the GC protection mechanism | Christopher Baines | 2023-04-17 |
| | | | | | Hopefully this is unused now. | ||
| * | Drop the delay for retrying uploads on failure | Christopher Baines | 2023-04-11 |
| | | |||
| * | Remove the crude alarm based timeout for submitting outputs | Christopher Baines | 2023-04-11 |
| | | | | | | This should be unnecessary now that there's progress on getting the I/O operations to timeout. | ||
| * | Add processor count to the agent status | Christopher Baines | 2023-03-24 |
| | | | | | This is useful when interpreting the load information. | ||
| * | Fix status load average handling | Christopher Baines | 2023-03-22 |
| | | | | | As the keys in JSON are strings. | ||
| * | Have agents send their status every 30 seconds | Christopher Baines | 2023-03-22 |
| | | |||
| * | Switch to using = for testing numerical equality | Christopher Baines | 2022-10-22 |
| | | | | | As I think I've been abusing eq?. | ||
| * | Tweak upload handling again | Christopher Baines | 2022-10-21 |
| | | | | | | | This partially reverts some recent changes. There should now be better handling of when all the bytes have been sent, but the hash hasn't yet been computed. | ||
| * | Log the file size and md5 hash on the agent side | Christopher Baines | 2022-10-21 |
| | | | | | When errors occur during upload. | ||
| * | Tweak output uploading code | Christopher Baines | 2022-10-19 |
| | | | | | Force sending the file from the start the first time the upload is attempted. | ||
| * | Use suspendable ports for the agent, with timeouts | Christopher Baines | 2022-10-16 |
| | | | | | | | | | This seems like a way of making the Guile internals for doing network I/O reliable. Currently, there are problems where things on the network timeout, but the Guile code for reading/writing just sits there, hung. This seems like it might help. | ||
| * | Throttle calling alarm | Christopher Baines | 2022-10-15 |
| | | | | | Just in case calling it frequently causes problems. | ||
| * | Reset the upload progress when appropriate | Christopher Baines | 2022-10-14 |
| | | |||
| * | Add more logging around file uploads | Christopher Baines | 2022-10-14 |
| | | |||
| * | Shorten timeout when submitting outputs | Christopher Baines | 2022-10-14 |
| | | | | | By resetting the timeout while the file is being uploaded. | ||
| * | Tweak output upload timeout | Christopher Baines | 2022-10-14 |
| | | | | | | Make it longer to avoid timeouts for large files, and increase the delay before restarting the upload. | ||
| * | Move retry in submit-output | Christopher Baines | 2022-04-09 |
| | | |||
| * | Add a timeout for submitting outputs | Christopher Baines | 2022-04-09 |
| | | |||
| * | Only use GC protection when gnutls won't internally retry | Christopher Baines | 2022-02-04 |
| | | |||
| * | Improve still more to send log message | Christopher Baines | 2021-12-22 |
| | | |||
| * | Improve logging when submitting outputs | Christopher Baines | 2021-12-22 |
| | | |||
| * | Only check the size of the file once when uploading | Christopher Baines | 2021-11-21 |
| | | |||
| * | Fix variable reference in submit-output | Christopher Baines | 2021-11-21 |
| | | |||
| * | Compress outputs outside of the upload slot | Christopher Baines | 2021-11-20 |
| | | | | | | So that the only thing taking place in the upload slot, is the actual upload, which should improve throughput. | ||
| * | Check if an output has been uploaded before trying to upload it | Christopher Baines | 2021-11-16 |
| | | | | | | | | | | | This can help if the output has been uploaded, but the hash isn't present, since trying to submit the build result will prompt for the output to be sent again, but it doesn't need to be, the agent just needs to wait. This is a little inelegant, maybe there needs to be some way for the agent to explicitly check for the hash to be computed, but I'm hoping these changes will help with uploading large outputs. | ||
| * | Fix the uri when calling coordinator-handle-failed-request | Christopher Baines | 2021-11-15 |
| | | |||
| * | Handle the case where there are no more bytes to send | Christopher Baines | 2021-11-15 |
| | | | | | | | | | | When submitting an output. This also fixes a regression in not passing report-bytes-sent on to call-with-streaming-http-request. I think this case where the agent is trying to send 0 bytes to the coordinator can happen when the last request to the coordinator times out, probably due to the computing of the hash taking so long. | ||
| * | Remove some test code | Christopher Baines | 2021-11-14 |
| | | |||
| * | Implement initial support for resuming HTTP uploads | Christopher Baines | 2021-11-14 |
| | | | | | | This means agents reattempting uploads don't have to start from scratch, and can instead pick up from what's already been uploaded to the coordinator. | ||
| * | Don't error for 404 responses in coordinator-http-request | Christopher Baines | 2021-11-14 |
| | | |||
| * | Don't error for responses with no body in coordinator-http-request | Christopher Baines | 2021-11-14 |
| | | |||
| * | Return to compressing outputs then sending them | Christopher Baines | 2021-11-07 |
| | | | | | | | | | | | | | | Trying to avoid the GnuTLS bindings breaking when the garbage collector runs is quite difficult, and the current approach isn't very effective. I want to try instead to support resuming partial uploads, as that should both help with the GnuTLS GC issue, as well as network interruptions in general. I think that approach is going to be easier with compressing the files up-front, so revert to doing that. This partially reverts commit 8258e9c8d9f729b2670a602c523c59847b676b1a. | ||
| * | Move retrying uploads out of the with-upload-slot region | Christopher Baines | 2021-08-07 |
| | | | | | | Such that the retry happens with a fresh slot (and the associated tracking information). | ||
| * | Support reporting bytes sent when submitting outputs | Christopher Baines | 2021-06-08 |
| | | |||