| Branch | Commit message | Author | Age |
|---|
| allow-concurrent-database-queries | database: Enable running up to 4 database queries at once. | Christopher Baines | 5 years |
| db-get-builds-performance-tuning | Begin tuning db-get-builds for performance | Christopher Baines | 5 years |
| handle-errors-in-worker-threads | utils: Handle errors in worker threads. | Christopher Baines | 5 years |
| improve-handling-of-rowids | Alter the Builds table to have an id field | Christopher Baines | 5 years |
| sqlite-statement-reset-fixes | Fix using sqlite-reset in a few queries | Christopher Baines | 4 years |
| support-publishing-build-events | Support publishing evaluation events | Christopher Baines | 5 years |
| support-publishing-build-events-minimal | Support publishing evaluation events | Christopher Baines | 5 years |
| support-returning-build-information-by-output | Support returning build information by output. | Christopher Baines | 5 years |
| wip-govuk | WIP | Christopher Baines | 8 years |
| |
| |
| Age | Commit message | Author |
|---|
| 2020-01-25 | database: Enable running up to 4 database queries at once.•••The number of threads is copied from bin/cuirass.in. When you have at least
two processors, this will allow database queries to be executed in parallel.
With some crude testing using the Apache HTTP server benchmarking tool (ab
from the httpd package), the max request latency does seem to drop when
multiple threads are used, especially when the database queries are slow (I
tested by adding usleep to the worker thread code).
* src/cuirass/database.scm (with-database): Pass #:parallelism to
make-worker-thread-channel.
allow-concurrent-database-queries | Christopher Baines |
| 2020-01-25 | Enable make-worker-thread-channel to create multiple worker threads.•••This will allow running multiple threads, that all listen on the same channel,
enabling processing multiple jobs at one time.
* src/cuirass/utils.scm (make-worker-thread-channel): Add a #:parallelism
argument, and create as many threads as the given parallelism.
| Christopher Baines |
| 2020-01-25 | Adjust make-worker-thread-channel to take an initializer.•••While this is a generic method, and initializer function will give the
flexibility required to create multiple worker threads for performing SQLite
queries, each with it's own database connection (as a result of calling the
initializer once for each thread). Without this change, they'd all have to use
the same connection, which would not work.
* src/cuirass/utils.scm (make-worker-thread-channel): Change procedure to take
an initializer, rather than arguments directly.
* src/cuirass/database.scm (with-database): Adjust to call
make-worker-thread-channel with an initializer.
* tests/database.scm (db-init): Change to use make-worker-thread-channel
initializer.
* tests/http.scm (db-init): Change to use make-worker-thread-channel
initializer.
| Christopher Baines |
| 2020-01-25 | utils: Change critical section terminology to worker threads.•••As far as I'm aware, it's necessary to use a separate thread for interacting
with SQLite as one of the threads used for fibers will be blocked while the
SQLite query is running.
This doesn't mean all queries have to be executed one at a time though,
providing the queries are executed outside the threads used by fibers, and a
single connection isn't used in multiple threads.
These changes start to move in this direction, first by just changing the
terminology.
* src/cuirass/base.scm (clear-build-queue, cancel-old-builds): Change
with-db-critical-section to with-db-worker-thread.
* src/cuirass/database.scm (with-db-critical-section): Rename syntax rule to
with-db-worker-thread.
(db-add-input, db-add-checkout, db-add-specification, db-remove-specification,
db-get-inputs, db-get-specification, db-add-evaluation,
db-set-evaluations-done, db-set-evaluation-done, db-add-derivation-output,
db-add-build, db-update-build-status!, db-get-output, db-get-outputs,
db-get-builds-by-search, db-get-builds, db-get-build derivation-or-id,
db-add-event, db-get-events, db-delete-events-with-ids-<=-to,
db-get-pending-derivations, db-get-checkouts, db-get-evaluations,
db-get-evaluations-build-summary, db-get-evaluations-id-max,
db-get-evaluation-summary, db-get-builds-query-min, db-get-builds-query-max,
db-get-builds-min, db-get-builds-max, db-get-evaluation-specification): Change
from using with-db-critical-section to
with-db-worker-thread.
(with-database): Change syntax rule to use make-worker-thread-channel,
renaming from make-critical-section.
* src/cuirass/utils.scm (%critical-section-args): Rename parameter to
%worker-thread-args.
(make-critical-section): Rename to make-worker-thread-channel, and adjust
parameter and docstring.
(call-with-critical-section): Rename to call-with-worker-thread and adjust
parameter.
(with-critical-section): Rename to with-worker-thread, and adjust to call
call-with-worker-thread.
* tests/database.scm (db-init): Use make-worker-thread-channel rather than
make-critical-section.
* tests/http.scm (db-init): Use make-worker-thread-channel rather than
make-critical-section.
| Christopher Baines |
| 2020-01-25 | Alter the Builds table to have an id field•••The internal rowid's are used for builds as you can request builds by using
the rowid in the URL.
The motivation here is to enable running VACUUM operations in SQLite, without
risking the rowid's for Builds changing. It would be bad if they change, as
they're used in the URL's for builds.
* src/schema.sql (Builds): Add id column.
* src/curiass/dataabse.scm (db-add-build): Change PRIMARYKEY constraint to
UNIQUE constraint.
* src/sql/upgrade-6.sql: New file.
* Makefile.am (dist_sql_DATA): Add it.
improve-handling-of-rowids | Christopher Baines |
| 2020-01-24 | database: Don't return rowid from db-add-input.•••As it is unused from where db-add-input is called.
* src/cuirass/database.scm (db-add-input): Don't call and
return (last-insert-rowid).
| Christopher Baines |
| 2020-01-18 | build: Allow builds with Guile 3.0.•••* configure.ac: Add "3.0" to 'GUILE_PKG'.
* README: Mention it.
| Ludovic Courtès |
| 2020-01-18 | http: Add missing import of (ice-9 threads) in test.•••Using the core binding for 'call-with-new-thread' was deprecated in 2.2
and is removed in 3.0.
* tests/http.scm: Use (ice-9 threads).
| Ludovic Courtès |
| 2020-01-16 | Support returning build information by output.•••Being able to take a derivation and query the build information is useful, but
in cases where there are multiple derivations that produce the same outputs,
the probability of getting the data back from Cuirass is reduced.
This is because Cuirass might not have build the exact derivation you have,
but a different derivation that produces the same outputs (this can commonly
happen when a related fixed output derivation changes).
Cuirass doesn't store derivations if they produce the same outputs as a
derivation it already knows about, so it can't determine if this is the
case. Therefore, provide a way of querying build results by output, rather
than derivation.
The motivation behind this is to make it easier to import build information in
to the Guix Data Service.
* src/cuirass/database.scm (db-get-output): New procedure.
* src/cuirass/http.scm (respond-output-not-found): New procedure.
(request-path-components): Handle /output/… requests.
* doc/cuirass.texi (Build information): Mention that you can get build
information by output.
| Christopher Baines |
| 2020-01-16 | Support publishing evaluation events•••* src/cuirass/database.scm (db-add-evaluation): Record the creation of new
evaluations as events.
(db-set-evaluation-done): Record when evaluations finish as an event.
* src/cuirass/http.scm (url-handler): Add a new /api/evaluation-events page.
| Christopher Baines |
| [...] |
| |
| Clone |
|---|
| https://git.cbaines.net/git/guix/cuirass |
| git@git.cbaines.net:guix/cuirass |