| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
Such that the retry happens with a fresh slot (and the associated tracking
information).
|
| | |
|
| |
|
|
|
| |
Since time has been spent building them, so wait longer before giving up
submitting the outputs.
|
| | |
|
| |
|
|
|
|
|
|
| |
Don't compress then send, since I think compression can be slower than
sending, so doing both at the same time is probably faster. Add
make-chunked-output-port* which might be more efficient than the Guile chunked
output port, will disable garbage collection to avoid issues with GnuTLS and
will try to force the garbage collector to run if there's garbage building up.
|
| | |
|
| |
|
|
|
| |
Since the gc breaking gnutls problem can occur for these requests probably as
well.
|
| |
|
|
| |
I think this works better.
|
| |
|
|
|
|
|
|
|
| |
This was put in to try and prevent the crashes inside gnutls, but was
ineffective since the actual trigger for the issues is garbage collection,
rather than parallel requests.
There might be some benefit from limiting request parallelism in the future,
but that can be thought through then.
|
| |
|
|
|
|
|
|
|
| |
Guile's garbage collector interferes with Guile+gnutls, which means that
sending files while the garbage collector is active is difficult.
These changes try to work around this by disabling the garbage collector just
as the data is being written, then enabling it again. I think this helps to
work around the issue.
|
| |
|
|
|
| |
Prefer upfront compression, as this might reduce GC activity while sending the
data.
|
| | |
|
| |
|
|
| |
Just use it when uploading files.
|
| |
|
|
| |
As I think this might make it faster.
|
| |
|
|
| |
This can happen if the request doesn't arrive in chunks.
|
| |
|
|
| |
I think this can happen if the log doesn't arrive as a chunked HTTP request.
|
| |
|
|
|
| |
I'm seeing mmap(PROT_NONE) failed crashes, and maybe these metrics will help
in understanding what's going on.
|
| |
|
|
|
|
| |
This avoids the need to create agents upfront, which could be useful when
creating many childhurd VMs or using scheduling tools to dynamically run
agents.
|
| |
|
|
| |
I've see the process hang on the hurd, and I think this might help.
|
| | |
|
| |
|
|
| |
I'm seeing this pull in sqlite3 unnecessarily on the hurd.
|
| |
|
|
|
|
|
|
|
|
|
| |
With with-exception-handler being called with #:unwind? #f (implicitly). This
breaks Guile internals used by (backtrace) [1], meaning you get a different
exception/backtrace when Guile itself breaks.
This should avoid the "string->number: Wrong type argument in position
1 (expecting string): #f" exception I've been haunted by for the last year.
1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46009
|
| | |
|
| | |
|
| |
|
|
|
| |
This is untested, but might be quite cool for running a single agent instance
of the build coordinator, all in one process.
|
| |
|
|
| |
As this code should be in the coordinator.
|
| | |
|
| |
|
|
| |
This will allow adding more agent messaging approaches.
|
| |
|
|
| |
So that the agent spends less time waiting.
|
| |
|
|
|
|
|
| |
Some parts of this were quite slow with anything other than a small database,
so instead of doing slow queries on every request, do some slow queries to
setup the metrics, and then change them as part of the regular changes to the
database.
|
| |
|
|
|
| |
I particularly want to monitor the WAL growth, as I don't think SQLite's usual
approach to keeping the size down is sufficient.
|
| |
|
|
| |
As it's shorter, and this keeps the logging neat.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Previously, an agent could end up fetching builds from the coordinator, but
not receiving the response, say because of a network issue or timeout. When it
retries, it would fetch even more builds, and there would be some allocated to
it, but that it doesn't know about.
These changes attempt to make fetching builds more idempotent, rather than
returning the new allocated builds, it returns all the builds, and rather than
requesting a number of new builds, it's the total number of allocated builds
that is specified.
|
| |
|
|
|
| |
Rather than polling the database every second, use some condition variables to
wake threads when there's probably an event.
|
| | |
|
| |
|
|
| |
This will help track CPU time, as well as restarts/crashes.
|
| |
|
|
|
| |
When submitting builds. The agent will now retry the relevant thing, like
uploading the log file if the coordinator says that still needs doing.
|
| |
|
|
|
| |
Things like the agent not having the log file, or an output. This will allow
the agent to actually retry the relevant thing.
|
| | |
|
| |
|
|
|
|
| |
build-log-file-location replaces build-log-file-exists? as it doesn't always
return a boolean, it also changes to return an absolute filepath for the log
file if it exists, as this will be easier to use.
|
| | |
|
| | |
|
| |
|
|
| |
So that the client part doesn't depend on fibers.
|
| |
|
|
|
| |
To start making it possible to use the agent, without having to load anything
related to fibers (as it doesn't work on the hurd yet).
|
| | |
|
| | |
|
| |
|
|
| |
Which was extracted from the Guix Build Coordinator.
|
| |
|
|
| |
Rather than the lzlib module within Guix.
|