aboutsummaryrefslogtreecommitdiff
path: root/guix-build-coordinator/agent-messaging/http.scm
Commit message (Collapse)AuthorAge
* Actually use non-blocking ports for network requestsChristopher Baines2024-03-02
| | | | In most places at least.
* Change retry-on-error to take #:ignore and #:no-retryChristopher Baines2024-01-12
| | | | And change #:ignore to better reflect ignoring the exception.
* Don't call report-bytes-hashed with #fChristopher Baines2023-05-24
| | | | Just log the line instead.
* Show backtraces for perform-upload errorsChristopher Baines2023-05-23
|
* Tweak logging around agent output submissionChristopher Baines2023-05-22
|
* Fix retry-times not always being setChristopher Baines2023-05-11
|
* Clean up some handling of uploads for agentsChristopher Baines2023-05-11
| | | | This commit should correct the progress reporting on partial uploads.
* Have agents report on the progress of the coordinator hashing outputsChristopher Baines2023-05-11
| | | | Otherwise it looks like the upload should finish, but hasn't.
* Tweak retrying for status update requestsChristopher Baines2023-05-10
| | | | | Don't retry status updates many times, since the information will be more out of date each time.
* Change submit-output to not spend so much time waitingChristopher Baines2023-05-08
| | | | | Make use of the coordinator trying to avoid the connection timing out. This should improve things when the coordinator is restarted or crashes.
* Stop using chunked transfers for file uploadsChristopher Baines2023-05-08
| | | | | As the amount of data to upload is known, this is unnecessary complexity and overhead.
* Add some comments about streaming http requestsChristopher Baines2023-05-05
|
* Include system uptime in the agent status informationChristopher Baines2023-05-05
| | | | As I've found this useful in spotting systems which have problems.
* Remove now redundant logging around gc protectionChristopher Baines2023-05-03
|
* Stop monitoring uploads through the chunked output portChristopher Baines2023-04-30
| | | | Use the dump-port* progress reporter instead.
* Deallocate canceled builds from agents when they startupChristopher Baines2023-04-21
|
* Remove the GC protection mechanismChristopher Baines2023-04-17
| | | | Hopefully this is unused now.
* Drop the delay for retrying uploads on failureChristopher Baines2023-04-11
|
* Remove the crude alarm based timeout for submitting outputsChristopher Baines2023-04-11
| | | | | This should be unnecessary now that there's progress on getting the I/O operations to timeout.
* Add processor count to the agent statusChristopher Baines2023-03-24
| | | | This is useful when interpreting the load information.
* Fix status load average handlingChristopher Baines2023-03-22
| | | | As the keys in JSON are strings.
* Have agents send their status every 30 secondsChristopher Baines2023-03-22
|
* Switch to using = for testing numerical equalityChristopher Baines2022-10-22
| | | | As I think I've been abusing eq?.
* Tweak upload handling againChristopher Baines2022-10-21
| | | | | | This partially reverts some recent changes. There should now be better handling of when all the bytes have been sent, but the hash hasn't yet been computed.
* Log the file size and md5 hash on the agent sideChristopher Baines2022-10-21
| | | | When errors occur during upload.
* Tweak output uploading codeChristopher Baines2022-10-19
| | | | Force sending the file from the start the first time the upload is attempted.
* Use suspendable ports for the agent, with timeoutsChristopher Baines2022-10-16
| | | | | | | | This seems like a way of making the Guile internals for doing network I/O reliable. Currently, there are problems where things on the network timeout, but the Guile code for reading/writing just sits there, hung. This seems like it might help.
* Throttle calling alarmChristopher Baines2022-10-15
| | | | Just in case calling it frequently causes problems.
* Reset the upload progress when appropriateChristopher Baines2022-10-14
|
* Add more logging around file uploadsChristopher Baines2022-10-14
|
* Shorten timeout when submitting outputsChristopher Baines2022-10-14
| | | | By resetting the timeout while the file is being uploaded.
* Tweak output upload timeoutChristopher Baines2022-10-14
| | | | | Make it longer to avoid timeouts for large files, and increase the delay before restarting the upload.
* Move retry in submit-outputChristopher Baines2022-04-09
|
* Add a timeout for submitting outputsChristopher Baines2022-04-09
|
* Only use GC protection when gnutls won't internally retryChristopher Baines2022-02-04
|
* Improve still more to send log messageChristopher Baines2021-12-22
|
* Improve logging when submitting outputsChristopher Baines2021-12-22
|
* Only check the size of the file once when uploadingChristopher Baines2021-11-21
|
* Fix variable reference in submit-outputChristopher Baines2021-11-21
|
* Compress outputs outside of the upload slotChristopher Baines2021-11-20
| | | | | So that the only thing taking place in the upload slot, is the actual upload, which should improve throughput.
* Check if an output has been uploaded before trying to upload itChristopher Baines2021-11-16
| | | | | | | | | | This can help if the output has been uploaded, but the hash isn't present, since trying to submit the build result will prompt for the output to be sent again, but it doesn't need to be, the agent just needs to wait. This is a little inelegant, maybe there needs to be some way for the agent to explicitly check for the hash to be computed, but I'm hoping these changes will help with uploading large outputs.
* Fix the uri when calling coordinator-handle-failed-requestChristopher Baines2021-11-15
|
* Handle the case where there are no more bytes to sendChristopher Baines2021-11-15
| | | | | | | | | When submitting an output. This also fixes a regression in not passing report-bytes-sent on to call-with-streaming-http-request. I think this case where the agent is trying to send 0 bytes to the coordinator can happen when the last request to the coordinator times out, probably due to the computing of the hash taking so long.
* Remove some test codeChristopher Baines2021-11-14
|
* Implement initial support for resuming HTTP uploadsChristopher Baines2021-11-14
| | | | | This means agents reattempting uploads don't have to start from scratch, and can instead pick up from what's already been uploaded to the coordinator.
* Don't error for 404 responses in coordinator-http-requestChristopher Baines2021-11-14
|
* Don't error for responses with no body in coordinator-http-requestChristopher Baines2021-11-14
|
* Return to compressing outputs then sending themChristopher Baines2021-11-07
| | | | | | | | | | | | | Trying to avoid the GnuTLS bindings breaking when the garbage collector runs is quite difficult, and the current approach isn't very effective. I want to try instead to support resuming partial uploads, as that should both help with the GnuTLS GC issue, as well as network interruptions in general. I think that approach is going to be easier with compressing the files up-front, so revert to doing that. This partially reverts commit 8258e9c8d9f729b2670a602c523c59847b676b1a.
* Move retrying uploads out of the with-upload-slot regionChristopher Baines2021-08-07
| | | | | Such that the retry happens with a fresh slot (and the associated tracking information).
* Support reporting bytes sent when submitting outputsChristopher Baines2021-06-08
|
generated by cgit v1.2.3 (git 2.45.0) at 2024-11-23 18:49:23 +0000