aboutsummaryrefslogtreecommitdiff
path: root/bin
Commit message (Collapse)AuthorAge
* Use the writer worker for all write queries.Mathieu Othacehe2020-10-15
| | | | | | | | | | | | | | | | | * .dir-locals.el: Add "with-queue-writer-worker". * bin/cuirass.in: Modify "with-queue-writer-worker" scope to include the web-server operations. * src/cuirass/database.scm (with-db-writer-worker-thread): Export it. (with-db-writer-worker-thread/force): New macro. (db-add-input, db-add-checkout, db-add-specification, db-remove-specification, db-add-evaluation, db-abort-pending-evaluations, db-set-evaluation-status, db-set-evaluation-time, db-add-output, db-add-build-product, db-add-event, db-delete-events-with-ids-<=-to): Use "with-db-writer-worker-thread" or "with-db-writer-worker-thread/force" instead of "with-db-worker-thread". * src/cuirass/metrics.scm (db-update-metrics): Ditto. * tests/database.scm ("db-init"): Set "%db-writer-channel". * tests/http.scm ("db-init"): Ditto. * tests/metrics.scm ("db-init"): Ditto.
* Queue write operations.Mathieu Othacehe2020-10-14
| | | | | | | | | | | | | | | | | | | | | | | | | SQLite only allows one concurrent write query operation. Having multiple database workers calling "db-update-build-status!", will thus increase worker starvation. Every write operation will also be done is a single transaction. For those reasons, create a database worker dedicated to write queries. Have this worker queue work and issue all the queued work queries in a single transaction. * .dir-locals.el: Add with-db-writer-worker-thread. * src/cuirass/database.scm (with-queue-writer-worker): Rename "with-registration-workers" macro. (%db-writer-channel): Rename "%db-registration-channel" variable. (with-queue-writer-worker): Rename "with-registration-workers". (db-register-builds): Use "with-db-writer-worker-thread" instead of "with-db-registration-worker-thread". (db-update-build-status!): Ditto * src/cuirass/utils.scm (make-worker-thread-channel): Add "queue-size" and "queue-proc" arguments. (call-with-worker-thread): Add "options" argument. * bin/cuirass.in (main): Use "with-queue-writer-worker" instead of "with-registration-workers". Modify the macro scope to include all the possible write operations.
* Add evaluation database workers.Mathieu Othacehe2020-10-01
| | | | | | | | | | | | | | Evaluation registration involves running a large number of SQL queries. This can cause some database worker starvation as well as some contention. To avoid this issue, spawn database workers dedicated to evaluation registration. * src/cuirass/database.scm (%db-registration-channel): New variable. (with-db-registration-worker-thread, with-registration-workers): New macros. (with-db-worker-thread-no-timeout): Remove it. (db-register-builds): Run registration in dedicated database workers using "with-db-registration-worker-thread" macro. * bin/cuirass.in (main): Spawn database registration workers by calling "with-registration-workers" macro.
* Add watchdog support.Mathieu Othacehe2020-09-29
| | | | | | | * src/cuirass/watchdog.scm: New file. * Makefile.am (dist_pkgmodule_DATA): Add it. * src/cuirass/utils.scm (with-timeout, get-message-with-timeout): Export them. * bin/cuirass.in (main): Start the watchdog.
* Do not raise an exception at gcroot directory creation.Mathieu Othacehe2020-09-24
| | | | | | | | | On fresh Guix System installations, Cuirass will not be allowd to create its own profile directory. Instead this should be take care of by guix-daemon when Cuirass will initiate its first connection to it. * bin/cuirass.in (main): Do not raise an exception on gcroot directory creation failure.
* Add SQL query logging support.Mathieu Othacehe2020-09-24
| | | | | | | | | * bin/cuirass.in (show-help): Document "--log-queries" option. (%options): Add it. (main): Enable query logging if the above option is set. * src/cuirass/database.scm (db-log-queries): New procedure. * src/cuirass/logging.scm (query-logging-port): New parameter. (log-query): New procedure.
* Log metrics update duration.Mathieu Othacehe2020-09-16
| | | | * bin/cuirass.in (main): Log metrics update duration.
* Add metrics support.Mathieu Othacehe2020-09-14
| | | | | | | | | | | | | | * Makefile.am: Add "upgrade-11.sql", "chart.js" and "metrics.js". * bin/cuirass.in (main): Add 'metrics fiber. * src/cuirass/http.scm (%file-white-list): Add "js/chart.js". (url-handler): Add "/metrics" route. * src/cuirass/metrics.scm: New file. * src/cuirass/templates.scm (html-page): Add a "Global metrics" sub-menu to "Status" dropdown menu. (make-line-chart, global-metrics-content): New procedures. * src/schema.sql (Metrics): New table. * src/sql/upgrade-11.sql: New file. * src/static/js/chart.js: New file.
* Add a status field to Evaluation table.Mathieu Othacehe2020-09-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The Evaluation table currently has an 'in_progress' field. Distinction between succeeded and failed evaluations are based on the presence of Builds records for the evaluation. It it also not possible to distinguish aborted evaluations from failed evaluations. Rename 'in_progress' field to 'status'. The 'status' field can be equal to 'started', 'succeeded', 'failed' or 'aborted'. * src/cuirass/database.scm (evaluation-status): New exported enumeration. (db-set-evaluations-done, db-set-evaluation-done): Remove them. (db-abort-pending-evaluations, db-set-evaluation-status): New exported procedures. (db-add-evaluation, db-get-builds, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluation-summary): Adapt to use 'status' field instead of 'in_progress' field. * src/cuirass/templates.scm (evaluation-badges): Ditto. * src/schema.sql (Evaluations): Rename 'in_progress' field to 'status'. * src/sql/upgrade-10.sql: New file. * bin/cuirass.in (main): Use "db-abort-pending-evaluations" instead of "db-set-evaluations-done". * src/cuirass/base.scm (evaluate): Use "db-set-evaluation-status" instead of "db-set-evaluations-done". (build-packages): Use "db-set-evaluation-status" instead of "db-set-evaluation-done". * tests/database.scm (sqlite-exec): Adapt accordingly. * tests/http.scm (evaluations-query-result): Ditto.
* web: Add monitoring fiber.Mathieu Othacehe2020-07-31
| | | | * bin/cuirass.in (main): Add monitoring fiber in web mode.
* cuirass: Perform some database "optimization" at startup.Christopher Baines2020-05-25
| | | | | | | | | | | | | | Add a "optimize" step that occurs when starting up the main Curiass process. Currently this does two things, but could be extended to do more. The "PRAGMA optimize;" command prompts SQLite to ANALYZE tables where that might help. The "PRAGMA wal_checkpoint(TRUNCATE);" command has SQLite process any unprocessed changes from the WAL file, then truncate it to 0 bytes. I've got no data to suggest this helps with performance, but I'm hoping that going from a large WAL file to a small one occasionally might be useful. * src/cuirass/database.scm (db-optimize): New procedure. * bin/cuirass.in (main): Run it.
* Support publishing build eventsChristopher Baines2020-01-16
| | | | | | | | | | | | | | | | | | | | | | | | | Add a table to store events, which have a type and a JSON blob. These can be used to record changes, this commit inserts events when new builds are created, and when the status of builds change. The EventsOutbox table is then used to track when events have been sent out. This is done through the new cuirass-send-events script. * Makefile.am (bin_SCRIPTS): Add bin/cuirass-send-events. .gitignore: Add bin/cuirass-send-events. (dist_pkgmodule_DATA): Add src/cuirass/send-events.scm. (dist_sql_DATA): Add src/sql/upgrade-5.sql. (EXTRA_DIST): bin/cuirass-send-events.in. (bin/cuirass-send-events): New rule. * bin/cuirass-send-events.in: New file. * src/cuirass/send-events.scm: New file. * src/sql/upgrade-5.sql: New file. * src/cuirass/database.scm (changes-count): New procedure. (db-update-build-status!): Call db-add-event after updating the build status. (db-add-event): New procedure. (db-add-build): Insert an event when a new build is inserted. (db-delete-events-with-ids-<=-to): New procedure. * src/schema.sql (Events): New table.
* bin: Remove redundant lambda.Ricardo Wurmus2019-10-30
| | | | * bin/cuirass.in (main): Remove redundant lambda wrapping in for-each.
* Separate web interface.Ricardo Wurmus2019-08-10
| | | | | | | * bin/cuirass.in (show-help): Document "--web" option. (%options): Default to running without web interface. (main): Either run the web interface or build packages. * README: Mention the "--web" option.
* base: Register GC roots for build results.Ludovic Courtès2018-11-07
| | | | | | | | | | | | | Fixes <https://bugs.gnu.org/33124>. * src/cuirass/base.scm (%gc-root-directory, %gc-root-ttl): New variables. (gc-root-expiration-time, register-gc-root): New procedures. (handle-build-event)[gc-roots]: New procedure. Upon 'build-succeeded' events, call 'register-gc-root' and 'maybe-remove-expired-cache-entries'. * bin/cuirass.in (show-help, %options): Add '--ttl'. (main): Parameterize %GC-ROOT-TTL. Create %GC-ROOT-DIRECTORY. * doc/cuirass.texi (Invocation): Document '--ttl'.
* database: Add a Checkouts table.Clément Lassieur2018-08-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
* database: Serialize all database accesses in a thread.Clément Lassieur2018-08-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes <https://bugs.gnu.org/32234>. * bin/cuirass.in (main): Keep only one WITH-DATABASE call around all fibers. Remove all DB arguments. * src/cuirass/base.scm (evaluate, update-build-statuses!, spawn-builds, handle-build-event, build-packages): Remove all DB arguments. (clear-build-queue, cancel-old-builds): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (restart-builds): Remove the NON-BLOCKING call, remove all DB arguments. (process-specs): Remove all DB arguments, remove the WITH-DATABASE call. * src/cuirass/database.scm (%db-channel): New parameter. (with-db-critical-section): New macro. (db-add-input, db-add-specification, db-get-inputs, db-get-specifications, db-add-evaluation, db-add-build, db-update-build-status!, db-get-outputs, db-get-builds, db-get-build, db-get-pending-derivations, db-get-stamp, db-add-stamp, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluations-id-min, db-get-evaluations-id-max, db-get-builds-min, db-get-builds-max): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (with-database): Wrap BODY in PARAMETERIZE form that sets %DB-CHANNEL to the channel returned by MAKE-CRITICAL-SECTION. * src/cuirass/http.scm (handle-build-request, handle-builds-request): Remove all DB arguments. (url-handler): Remove all DB arguments, remove the DB-CHANNEL state, remove the WITH-CRITICAL-SECTION calls. (run-cuirass-server): Remove the DB arguments, remove the MAKE-CRITICAL-SECTION call. * src/cuirass/utils.scm (make-critical-section): Replace SPAWN-FIBER with CALL-WITH-NEW-THREAD. Wrap body in PARAMETERIZE form that clears CURRENT-FIBER. * tests/database.scm (with-temporary-database, "db-add-specification", "db-add-build", "db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Remove the DB arguments. ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("database"): Set %DB-CHANNEL to #f during cleanup. * tests/http.scm ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("cuirass-run", "fill-db"): Remove the DB arguments. ("http"): Set %DB-CHANNEL to #f during cleanup.
* Add support for multiple inputs.Clément Lassieur2018-07-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql. * bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that was used afterwards as %GUIX-PACKAGE-PATH. * bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path, spec-package-path, format-checkouts): New procedures. (%not-colon): Remove variable. (main): Take the load path, package path and PROC from the checkouts that result from the inputs. Format the checkouts before sending them to the procedure. Remove the LOAD-PATH argument. * doc/cuirass.texi (Overview, Database schema): Document the changes. * examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm, hello-subset.scm, random.scm}: Adapt to the new specification format. * examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT. (package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle the new format of its return value. * examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT. Rename the checkout from 'random (which is a specification) to 'cuirass (which is a checkout resulting from an input). * src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC to INPUT. Return a checkout object instead of returning two values. (evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object instead of getting it from "evaluate". (compile?, fetch-inputs, compile-checkouts): New procedures. (process-specs): Fetch all inputs instead of only fetching one repository. The result of that fetching operation is a list of CHECKOUTS whose COMMITS are used as a STAMP. (%guix-package-path, set-guix-package-path): Remove them. * src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures. (db-add-specification, db-get-specifications): Adapt to the new specification format. Add/get all inputs as well. (db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space separated commit hashes. (db-get-builds): Rename REPO_NAME to NAME. (db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP. (db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return value. (db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS. * src/cuirass/utils.scm (%non-blocking): Export it. * src/schema.sql (Inputs): New table that refers to the Specifications table. (Specifications): Move input related fields to the Inputs table. Rename REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE. Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to the Inputs table. (Stamps): Rename REPO_NAME to NAME. (Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS. (Specifications_index): Replace with Inputs_index. * src/sql/upgrade-1.sql: New file. * tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to the new specifications format. Rename REVISION to COMMITS. * tests/http.scm (evaluations-query-result, fill-db): Idem.
* evaluate: Use a generic key to identify Cuirass arguments.Clément Lassieur2018-06-18
| | | | | | | | | | | | | | | | | So that Cuirass specifications used to build 'guix-modular' can be named differently than "guix" and "guix-modular" (see Guix's build-aux/hydra/guix-modular.scm). The name is used as a primary key, so before that commit, it was also impossible to have several such specifications. This is a workaround, Cuirass should normally support several inputs per specification, as Hydra does. The specification's name would then only be used to identify it. See <https://bugs.gnu.org/31813>. * bin/evaluate.in (main): Replace custom NAME (passed to PROC) with 'guix'. Co-authored-by: Mathieu Othacehe <m.othacehe@gmail.com>
* Use the 2.2 'setvbuf' API style.Ludovic Courtès2018-04-09
| | | | | * bin/cuirass.in (main): Use the 2.2 'setvbuf' API style. * examples/guix-track-git.scm (current-error-port): Likewise.
* evaluate: Change '%load-path' once and for all.Ludovic Courtès2018-04-09
| | | | * bin/evaluate.in (main): Do not restore the original '%load-path'.
* evaluate: Leave GUILE_LOAD_PATH unchanged.Ludovic Courtès2018-04-09
| | | | | | | | | | | | Previously we'd systematically add the first argument to the search path. When that first argument was the empty string (as with the 'guix-modular' jobset), we'd thus add the current directory to the search path, even if the intent was to leave the load path unchanged. Furthermore, a3a7c09b06027bd30a96ae4607fa40bd790af840 changed load-path handling to be explicit in Scheme. * bin/evaluate.in: Remove GUILE_LOAD_PATH assignment.
* evaluate: Change directory to SOURCE.Ludovic Courtès2018-04-09
| | | | | * bin/evaluate.in (with-directory-excursion): Remove. (main): chdir to SOURCE.
* evaluate: Do not load Guix/Cuirass modules upfront.Ludovic Courtès2018-04-09
| | | | | | | | | | | | | | This avoids a situation whereby, when evaluating from a Guix checkout, we'd have already loaded slightly different and incompatible (guix …) modules. Hydra's 'hydra-eval-guile-jobs' implemented the same solution as in this patch already. * bin/evaluate.in: Remove use of (cuirass …) and (guix …) modules. (ref): New procedure. (with-directory-excursion): New macro. (main): Use 'ref'. Remove uses of Guix or Cuirass modules.
* evaluate: Honor the given load path.Ludovic Courtès2018-04-07
| | | | | * bin/evaluate.in (main): Prepend LOAD-PATH to '%load-path' for the dynamic extend of the 'primitive-load' call.
* evaluate: Really support 'build-derivations' RPCs.Ludovic Courtès2018-04-07
| | | | | | | Fixes a thinko introduced in 1872dd95253b4805a00bfe5dee8d1a0ed90af149. * bin/evaluate.in (main): Make sure 'real-build-things' is bound.
* evaluate: Really pass arguments like Hydra.Ludovic Courtès2018-04-06
| | | | | * bin/evaluate.in (main): Use SPEC's #:name as a key in ARGS. * examples/random-jobs.scm (make-random-jobs): Adjust accordingly.
* base: Let sqlite handle deduplication of the list of pending derivations.Ludovic Courtès2018-04-05
| | | | | | | | | | | | | | | | | | | | | | | | | Previously we would make a SQL query that would return many build jobs, and then call 'delete-duplicates' on that. This was extremely wasteful because the list of returned by the query was huge leading to a heap of several tens of GiB on a big database, and 'delete-duplicates' would lead to more GC and it would take ages. Furthermore, since 'delete-duplicates' is written in C as of Guile 2.2.3, it is uninterruptible from Fiber's viewpoint. Consequently, the kernel thread running the 'restart-builds' fiber would never schedule other fibers, which could lead to deadlocks--e.g., since fibers are scheduled on a circular shuffled list of kernel threads, once every N times, a web server fiber would be sent to that kernel thread and not be serviced. * src/cuirass/base.scm (shuffle-jobs): Remove. (shuffle-derivations): New procedure. (spawn-builds): Take a list of derivations instead of a list of jobs. (restart-builds): Remove 'builds' parameter. Remove 'delete-duplicates' call. Remove done/remaining partitioning. (build-packages): Adjust to pass 'spawn-builds' a list of derivations. * bin/cuirass.in (main): Remove computation of PENDING. Remove second parameter in call to 'restart-builds'.
* evaluate: Pass the file name and revision to the user procedure.Ludovic Courtès2018-04-05
| | | | | | | * bin/evaluate.in (main): Always pass an alist as the arguments to PROC, containing at least 'file-name' and 'revision'. * examples/random-jobs.scm (make-random-jobs): Display 'file-name' and 'revision' from ARGUMENTS.
* base: Make a writable copy of the checkout only when #:no-compile? is false.Ludovic Courtès2018-04-01
| | | | | | | | | | | | | | | This avoids copying things back and forth. * src/cuirass/base.scm (fetch-repository): Add #:writable-copy? parameter. Call 'make-writable-copy' when it's true. (copy-repository-cache): Remove. (make-writable-copy): New procedure. (evaluate): Add 'source' parameter and pass it to the 'evaluate' program. (process-specs): Define 'compile?'. Pass #:writable-copy? to 'fetch-repository'. Remove call to 'copy-repository-cache'. Remove computation of the checkout directory name. Pass CHECKOUT to 'evaluate'. * bin/evaluate.in (main): Replace 'cachedir' with 'source'. Remove computation of the checkout directory name.
* evaluate: Tolerate calls to 'build-things' during evaluations.Ludovic Courtès2018-03-28
| | | | * bin/evaluate.in (main): 'build-things' replacement no longer calls 'exit'.
* cuirass: Line-buffer stdout and stderr.Ludovic Courtès2018-03-25
| | | | * bin/cuirass.in (main): Add 'setvbuf' calls.
* base: Move database update from 'evaluate' process to the main process.Ludovic Courtès2018-03-01
| | | | | | | | | | | | Fixes <https://bugs.gnu.org/30618>. Reported by Andreas Enge <andreas@enge.fr>. * bin/evaluate.in (fill-job): Remove. (main): Remove 'database' command-line argument. Remove DB and its uses. Write an (evaluation EVAL JOBS) sexp. * src/cuirass/base.scm (evaluate)[augment-job]: New procedure. Use it. Adjust to read (evaluation EVAL JOBS) sexp. Call 'db-add-evaluation' and 'db-add-derivation'.
* cuirass: Clear the build queue when starting.Ludovic Courtès2018-02-14
| | | | | * src/cuirass/base.scm (clear-build-queue): New procedure. * bin/cuirass.in (main): Call it.
* cuirass: Add '--threads' and put an upper bound on the default.Ludovic Courtès2018-02-05
| | | | | * bin/cuirass.in (show-help, %options): Add "--threads". (main): Honor it. Pass #:parallelism to 'run-fibers'.
* cuirass: Log resource usage statistics regularly.Ludovic Courtès2018-01-29
| | | | | * src/cuirass/logging.scm (log-monitoring-stats): New procedure. * bin/cuirass.in (main): Add a fiber that calls it regularly.
* cuirass: Place web server fiber on its own core.Ludovic Courtès2018-01-27
| | | | | * bin/cuirass.in (main): Pass #:parallel? to 'spawn-fiber' for the web server.
* cuirass: Add 'essential-task' and wrap the main fibers in it.Ludovic Courtès2018-01-27
| | | | | * src/cuirass/utils.scm (essential-task): New procedure. * bin/cuirass.in (main): Wrap each fiber in 'essential-task'.
* cuirass: Reduce Fiber's tick rate.Ludovic Courtès2018-01-26
| | | | * bin/cuirass.in (main): Pass #:hz to 'run-fibers'.
* cuirass: Catch exceptions in the main fiber and stop everything.Ludovic Courtès2018-01-26
| | | | | | * bin/cuirass.in (main): Add 'exit-channel' and read from it. Catch exceptions in the main fiber and write to that channel upon error.
* cuirass: Log pending build request.Ludovic Courtès2018-01-26
| | | | * bin/cuirass.in (main): Print message while fetching pending builds.
* cuirass: Fix scope mismatch.Ludovic Courtès2018-01-23
| | | | | | | | | | Previously STORE would be closed on the first context switch, before 'restart-builds' has completed, leading to an error (writing to a closed port). * src/cuirass/base.scm (restart-builds): Remove 'store' parameter. Wrap body in 'with-store'. * bin/cuirass.in (main): Adjust accordingly.
* cuirass: Add missing import.Ludovic Courtès2018-01-23
| | | | * bin/cuirass.in: Import (guix store).
* base: Restart pending builds upfront.Ludovic Courtès2018-01-23
| | | | | | | | * src/cuirass/database.scm (db-get-builds)[format-where-clause]: Honor (status pending) filter. * src/cuirass/base.scm (restart-builds): New procedure. * bin/cuirass.in (main): Fetch pending builds. Start fiber that invokes 'restart-builds' on them.
* Call 'run-cuirass-server' outside of a fiber.Ludovic Courtès2018-01-23
| | | | | | | | This seems to fix a bug whereby the server would never process the FD returned by 'accept'. * bin/cuirass.in (main): Move 'run-cuirass-server' outside of 'spawn-fiber'.
* Introduce concurrency with Fibers.Ludovic Courtès2018-01-22
| | | | | | | | | | | | | | | * README: Mark Fibers as required. * configure.ac: Check for Guile 2.2 only. Check for (fibers). * bin/cuirass.in (main): Use (fibers). Run 'process-specs' and web server in separate fibers. * src/cuirass/base.scm (with-store): New macro. (non-blocking-port): New procedure. (evaluate): Use 'non-blocking-port'. Use 'read-string' followed by 'read'. (process-specs): Move 'db-add-stamp' right after 'string=?' comparison. Run evaluation and subsequent builds in a separate fiber. * src/cuirass/http.scm (run-cuirass-server): Pass 'fibers as the second argument to 'run-server'. Use 'log-message' instead of 'format'. * src/cuirass/database.scm (with-database): Remove 'dynamic-wind'.
* Initialize libgit's TLS certificate directory once and for all.Ludovic Courtès2018-01-22
| | | | | | * src/cuirass/base.scm (prepare-git): New procedure. Body moved from... (process-specs): ... here. Remove it. * bin/cuirass.in (main): Call 'prepare-git'.
* cuirass: Add --listen command line option.Jan Nieuwenhuizen2017-09-18
| | | | | | | | * bin/cuirass.in (show-help): Add help for --listen option. (%options): Add listen option. (main): Pass host to run-cuirass-server. * doc/cuirass.texi (Invocation): Add --listen option. * src/cuirass/http.scm (run-cuirass-server): Add named #:host parameter.
* cuirass: Store new information in database to prepare new HTTP APIMathieu Othacehe2017-09-08
| | | | | | | | | | | | | | | | | | | | | integration. * bin/evaluate.in (fill-job): New procedure. (main): Use it to fill informations (nix-name, system) that will later be added to database. * doc/cuirass.texi (Database)[Derivation]: Add system and nix_name fields. (Database)[Builds]: Add id, status, timestamp, starttime and stoptime fields. Remove output field. (Database)[Outputs]: New table describing the build outputs. * src/cuirass/base.scm (build-packages): Add new fields to build object before adding it to database. * src/cuirass/database.scm (db-get-build, db-get-builds): New procedures to get a build by id from database and a list of builds using filter parameters respectively. * src/schema.sql (Outputs) : New table. (Derivations): Add system and nix_name columns. (Builds): Remove output column and add id, status, timestamp, starttime and stoptime columns.
* cuirass: Add fallback parameter.Mathieu Othacehe2017-07-12
| | | | | | | | * src/cuirass/base.scm (%fallback?): New exported parameter. (process-specs)[set-build-options]: Pass fallback parameter. * bin/cuirass.in (%options): Add fallback. (main): Set %fallback? parameter. (show-help): Update.
generated by cgit v1.2.3 (git 2.50.1) at 2025-10-20 18:42:12 +0000