aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* WIPwip-upload-slotsChristopher Baines2021-06-08
|
* WIPChristopher Baines2021-06-08
|
* Ensure there's data in backtracesChristopher Baines2021-06-07
|
* Switch to a lower gc threshold in make-chunked-output-port*Christopher Baines2021-06-07
| | | | | A fixed threshold should make things line up when there's multiple ports in use.
* Refactor the code around work queuesChristopher Baines2021-05-31
|
* Retry more when sending outputsChristopher Baines2021-05-30
| | | | | Since time has been spent building them, so wait longer before giving up submitting the outputs.
* Don't pretend that chunked inputs have been readChristopher Baines2021-05-30
| | | | | If the connection closes before all the data's been received. Most of a nar file isn't useful.
* Improve the reveived output messageChristopher Baines2021-05-30
|
* Fix closing the custom chunked output portChristopher Baines2021-05-30
|
* Fix the make-chunked-output-port* implementationChristopher Baines2021-05-30
| | | | A broken one was committed previously.
* Further tweak sending chunked HTTP requestsChristopher Baines2021-05-29
| | | | | | | | Don't compress then send, since I think compression can be slower than sending, so doing both at the same time is probably faster. Add make-chunked-output-port* which might be more efficient than the Guile chunked output port, will disable garbage collection to avoid issues with GnuTLS and will try to force the garbage collector to run if there's garbage building up.
* Tweak how the GC protection works when closing the portChristopher Baines2021-05-28
| | | | Bring more stuff inside one with-gc-protection block.
* Increase the size of one of the buffers for sending filesChristopher Baines2021-05-28
| | | | As this reduces the GC disabling/enabling.
* Add more gc protectionChristopher Baines2021-05-28
|
* Use dyanmic-wind when changing the GC configurationChristopher Baines2021-05-28
| | | | | | | These procedures actually increment/decrement a counter, so gc-enable might not enable garbage collection if gc-disable has been called twice in a row. dynamic-wind should ensure that gc-enable is always called after gc-disable, even if thunk raises an exception for example.
* Don't automatically requeue jobsChristopher Baines2021-05-28
| | | | | | | | Even if they aren't requeued, the agent should learn about the job again from the coordinator. I'm mostly removing this, because I'm seeing agents seemingly process the same job twice at the same time, and I wonder if it's related.
* Add a space in coordinator-handle-failed-requestChristopher Baines2021-05-28
|
* Improve retrying for missing log filesChristopher Baines2021-05-28
|
* Have with-gc-protection return the thunk return values properlyChristopher Baines2021-05-28
|
* Use GC protection for normal requests to the coordinator as wellChristopher Baines2021-05-28
| | | | | Since the gc breaking gnutls problem can occur for these requests probably as well.
* Increase the buffer size for sending outputs and log filesChristopher Baines2021-05-28
| | | | I think this works better.
* Get rid of the request mutexChristopher Baines2021-05-28
| | | | | | | | | This was put in to try and prevent the crashes inside gnutls, but was ineffective since the actual trigger for the issues is garbage collection, rather than parallel requests. There might be some benefit from limiting request parallelism in the future, but that can be thought through then.
* Tune sending files over HTTPChristopher Baines2021-05-28
| | | | | | | | | Guile's garbage collector interferes with Guile+gnutls, which means that sending files while the garbage collector is active is difficult. These changes try to work around this by disabling the garbage collector just as the data is being written, then enabling it again. I think this helps to work around the issue.
* Fix datastore-find-derivation-for-outputChristopher Baines2021-05-27
|
* Fix part of the create_outputs migrationChristopher Baines2021-05-27
| | | | | It was completely broken, since it would just delete all entries from the unbuilt outputs table.
* Fix datastore-list-unbuilt-derivation-outputsChristopher Baines2021-05-27
|
* Fix datastore-list-build-outputs for the new outputs tableChristopher Baines2021-05-27
|
* Reduce the threshold for compressing nars on the flyChristopher Baines2021-05-26
| | | | | Prefer upfront compression, as this might reduce GC activity while sending the data.
* Remove stale log filesChristopher Baines2021-05-26
|
* Don't ignore premature termination for chunked requestsChristopher Baines2021-05-26
| | | | | | This code was copied from Guile, but this seems like a deficiency. I can't imagine a case where you'd be processing chunked data, and just want to pretend you've got to the end, when you haven't.
* Delay storing derivations in the databaseChristopher Baines2021-05-21
| | | | | Until actually storing the build, since the build might not actually be submitted if there's a build for those outputs already.
* Return #f from datastore-find-derivation-outputs if none are foundChristopher Baines2021-05-21
| | | | As this is clearer than '().
* Allow datastore-find-derivation-system to return #fChristopher Baines2021-05-21
| | | | If the derivation doesn't exist.
* Switch to a default 1min load average of 3/4 of the processor countChristopher Baines2021-05-21
| | | | As this will be a better default on machines with more cores.
* Drop the request mutex for most requestsChristopher Baines2021-05-21
| | | | Just use it when uploading files.
* Enable foreign key checkingChristopher Baines2021-05-21
| | | | I thought this would be on by default, but apparently not.
* Introduce a systems tableChristopher Baines2021-05-21
|
* Introduce an outputs tableChristopher Baines2021-05-21
|
* Line buffer the queue builds script outputChristopher Baines2021-05-14
| | | | So that the output is more consistent.
* Simplify the work queue loopChristopher Baines2021-05-14
|
* Buffer the network socket used for streaming http requestsChristopher Baines2021-05-13
| | | | As this helps improve throughput.
* Use a bigger buffer when uploading logsChristopher Baines2021-05-13
| | | | As I think this might make it faster.
* Increase the basic build allocation strategy default planned buildsChristopher Baines2021-04-27
| | | | | Mostly so the coordinator for building patches plans more builds at once, since the allocator is very slow at the moment.
* Stop agent threads only when 2 or more are idleChristopher Baines2021-04-23
| | | | | | If the jobs are really quick, I think the one running thread keeps stopping and starting, and that stops the agent starting more threads. I think this change might help.
* Handle receiving outputs as a bytevectorChristopher Baines2021-04-23
| | | | This can happen if the request doesn't arrive in chunks.
* Remove non-existent exportChristopher Baines2021-04-20
|
* Make some SQLite related improvementsChristopher Baines2021-04-20
| | | | | Don't keep database connections around forever as this relates to cached query plans, and also run the optimize pragma when closing connections.
* Support destructors and lifetimes for worker threadsChristopher Baines2021-04-20
|
* Support finding blocking builds not including cancelled onesChristopher Baines2021-04-16
| | | | As including cancelled ones can make the query much slower.
* Handle receiving logs as bytevectorsChristopher Baines2021-04-09
| | | | I think this can happen if the log doesn't arrive as a chunked HTTP request.
generated by cgit v1.2.3 (git 2.25.4) at 2024-05-02 01:26:24 +0000