summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* Revert "http: Process client connections really concurrently."Ludovic Courtès2018-02-14
| | | | | | | | This reverts commit c47dfdf82b4be62501a7932eaec4c124566a1829. Processing connections concurrently would require having separate database handles. See <https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00206.html>.
* database: Make 'db-add-derivation' idempotent.Ludovic Courtès2018-02-14
| | | | | | | | * src/cuirass/database.scm (db-add-derivation): Catch 'sqlite-error and handle SQLITE_CONSTRAINT_PRIMARYKEY. (SQLITE_CONSTRAINT_UNIQUE): New variable. * tests/database.scm ("database")["db-add-derivation"]: Add extra call to 'db-add-derivation'.
* http: Process client connections really concurrently.Ludovic Courtès2018-02-10
| | | | | | | | Before that, 'run-server' would force sequential processing of client requests one after another. * src/cuirass/http.scm (run-cuirass-server): Rewrite to use its own loop instead of 'run-server'.
* utils: 'non-blocking' forwards exceptions to the calling fiber.Ludovic Courtès2018-02-08
| | | | | * src/cuirass/utils.scm (%non-blocking): Forward exceptions to the calling fiber.
* database: Use argument binding in 'db-get-builds' queries.Ludovic Courtès2018-02-08
| | | | | | | | | That makes it safe from SQL injection. * src/cuirass/database.scm (db-get-builds): Rewrite to use question marks in SQL queries and binding through '%sqlite-exec'. * tests/database.scm ("database")["db-get-builds"]: Exercise 'WHERE' clauses.
* database: Handle binding directly in 'sqlite-exec'.Ludovic Courtès2018-02-08
| | | | | | | | | | | | | | | | | The new macro automatically takes care of inserting question marks in the SQL queries, which in turn guarantees that there are always as many question marks and arguments. * src/cuirass/database.scm (sqlite-exec): Rename to... (%sqlite-exec): ... this. (sqlite-exec/bind, sqlite-exec): New macros. (assq-refs): Remove. (db-add-specification): Use the new 'sqlite-exec' form. (db-get-specifications): Correctly deal with REV or TAG being #f. (db-add-derivation, db-get-derivation, db-add-evaluation) (db-add-build, db-update-build-status!, db-get-outputs) (db-get-build, db-get-stamp, db-add-stamp): Adjust to the new 'sqlite-exec' form.
* examples: random: Make derivations a bit more random.Ludovic Courtès2018-02-08
| | | | | * examples/random-jobs.scm (random-derivation): Tweak the computation of SEED. Actually use the new random state.
* base: Account for derivations built behind our back.Ludovic Courtès2018-02-08
| | | | | | | | | | Previously any derivation not directly built by Cuirass would be considered as failed because 'handle-build-event' wouldn't see any build event. Here we just make sure the build status recorded in the database corresponds to reality. * src/cuirass/base.scm (update-build-statuses!): New procedure. (spawn-builds): Call it after 'build-derivations&'.
* base: Fix computation of build success/failure.Ludovic Courtès2018-02-08
| | | | | | * src/cuirass/base.scm (build-packages)[register]: Make 'db-add-build' a tail call. Fix computation of 'outs'.
* examples: random: Make the job name a symbol.Ludovic Courtès2018-02-08
| | | | * examples/random-jobs.scm (make-job): Add '.' in #:job-name.
* database: 'db-update-build-status!' keeps stoptime unchanged when nothing new.Ludovic Courtès2018-02-08
| | | | | | | * src/cuirass/database.scm (db-update-build-status!): And "AND status != ?" in SQL queries. * tests/database.scm ("database")["db-update-build-status!"]: Add call to 'db-update-build-status!'.
* base: Explicitly enable #:print-build-trace.Ludovic Courtès2018-02-08
| | | | | | | | This is purely "defensive programming" since the default value for 'print-build-trace' is currently #t. * src/cuirass/base.scm (with-store): Pass #:print-build-trace to 'set-build-options'.
* database: Cache prepared statements.Ludovic Courtès2018-02-08
| | | | | | | Suggested by Danny Milosavljevic. * src/cuirass/database.scm (sqlite-exec): Pass #:cache? to 'sqlite-prepare'.
* database: Use 'sqlite-bind' to avoid SQL injection.Danny Milosavljevic2018-02-08
| | | | | | | | | | | | | | | | | | | * src/cuirass/database.scm (%sqlite-exec): Remove. (sqlite-exec): Turn back into a procedure. Use 'sqlite-bind'. Add 'normalize' procedure and use it. (db-add-specification, db-add-derivation, db-get-derivation) (db-add-evaluation, db-add-build, db-update-build-status!) (db-get-build, db-get-stamp, db-add-stamp): Use question marks in SQL queries. * src/cuirass/base.scm (build-packages)[register]: Make #:log non-false. * tests/database.scm (make-dummy-job): Add #:job-name, #:system, #:nix-name, and #:eval-id. This is necessary because 'sqlite-bind' would now translate #f to a real NULL (before it would translate to the string "#f"...), and would thus report violations of the non-NULL constraint. Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* base: Remove useless 'log-message' calls.Ludovic Courtès2018-02-05
| | | | | * src/cuirass/base.scm (build-packages): Remove useless 'log-message' calls.
* http: /jobsets returns a list of jobsets.Ludovic Courtès2018-02-05
| | | | | | | Reported by Danny Milosavljevic. * src/cuirass/http.scm (url-handler) <"jobsets"> Return the result of 'db-get-specifications' as-is, not just the car.
* base: Log the number of builds performed.Ludovic Courtès2018-02-05
| | | | | * src/cuirass/base.scm (spawn-builds): Log the number of builds performed, not the number of remaining builds.
* cuirass: Add '--threads' and put an upper bound on the default.Ludovic Courtès2018-02-05
| | | | | * bin/cuirass.in (show-help, %options): Add "--threads". (main): Honor it. Pass #:parallelism to 'run-fibers'.
* base: Work around Fibers I/O scheduling bug.Ludovic Courtès2018-01-30
| | | | | | | | | | | Works around <https://github.com/wingo/fibers/issues/19>. The effect in practice would be that we'd usually not reach the 'close-pipe' call in 'evaluate', leaving zombie processes behind us, never executing the continuation, and additionally spinning fast on a sequence of epoll_wait/epoll_ctl calls. * src/cuirass/base.scm <top level>: Monkey-patch (fibers internal).
* examples: Add 'random-jobs'.Ludovic Courtès2018-01-29
| | | | | * examples/random-jobs.scm, examples/random.scm: New files. * Makefile.am (nobase_dist_pkgdata_DATA): Add them.
* base: Make build log processing non-blocking.Ludovic Courtès2018-01-29
| | | | | | | | | | | | | | | | | We used to have 'build-derivations' write to the custom binary port returned by 'build-event-output-port'. However, custom binary ports constitute continuation barriers, thereby preventing fibers from being suspended. To make build log processing non-blocking, we therefore invert this inversion of control and use a suspendable I/O procedure, 'read-line/non-blocking', when reading the build log. * src/cuirass/base.scm (read-line/non-blocking, process-build-log) (build-derivations&): New procedures. (%newline, build-event-output-port): Remove. (spawn-builds): Use 'build-derivations&' instead of 'build-derivations' with 'build-event-output-port'.
* http: Evaluate DB requests in 'non-blocking'.Ludovic Courtès2018-01-29
| | | | | * src/cuirass/http.scm (handle-builds-request): Wrap 'db-get-builds' into 'non-blocking'.
* cuirass: Log resource usage statistics regularly.Ludovic Courtès2018-01-29
| | | | | * src/cuirass/logging.scm (log-monitoring-stats): New procedure. * bin/cuirass.in (main): Add a fiber that calls it regularly.
* http: /api/queue returns builds sorted by status.Ludovic Courtès2018-01-29
| | | | | | * src/cuirass/database.scm (db-get-builds)[format-order-clause]: Add 'status+submission-time'. * src/cuirass/http.scm (url-handler) <"queue">: Use it.
* base: 'spawn-builds' shuffles jobs.Ludovic Courtès2018-01-29
| | | | | * src/cuirass/base.scm (shuffle-jobs): New procedure. (spawn-builds): Use it.
* base: 'spawn-builds' logs the beginning of each new batch.Ludovic Courtès2018-01-29
| | | | * src/cuirass/base.scm (spawn-builds): Add 'log-message' call.
* build-aux: Add guile-fibers.Danny Milosavljevic2018-01-29
| | | | * build-aux/guix.scm (inputs): Add guile-fibers.
* examples: Use new API.Danny Milosavljevic2018-01-29
| | | | | * examples/guix-track-git.scm (package->git-tracked): Add STORE parameter. (guix-jobs): Pass value for STORE parameter.
* cuirass: Place web server fiber on its own core.Ludovic Courtès2018-01-27
| | | | | * bin/cuirass.in (main): Pass #:parallel? to 'spawn-fiber' for the web server.
* cuirass: Add 'essential-task' and wrap the main fibers in it.Ludovic Courtès2018-01-27
| | | | | * src/cuirass/utils.scm (essential-task): New procedure. * bin/cuirass.in (main): Wrap each fiber in 'essential-task'.
* http: Reject methods other than GET.Ludovic Courtès2018-01-26
| | | | | * src/cuirass/http.scm (url-handler): Check whether REQUEST's method is 'GET, and return 405 if not.
* http: Provide the correct values for 'finished' and 'busy'.Ludovic Courtès2018-01-26
| | | | | | | | This should placate 'guix-hydra-queued-builds', for instance, which checks these values to choose between "Running" and "Scheduled". * src/cuirass/http.scm (build->hydra-build): Provide the correct value for #:finished and #:busy.
* database: Open the database in "write-ahead log" mode.Ludovic Courtès2018-01-26
| | | | | | | Suggested by Ricardo Wurmus. * src/cuirass/database.scm (wal-mode): New procedure. (db-open): Use it.
* base: 'build-event-output-port' no longer reads past the buffer.Ludovic Courtès2018-01-26
| | | | | | | | | Fixes a bug whereby 'build-event-output-port' would sometimes read more than COUNT from BV, which would usually result in a 'decoding-error' exception from 'utf8->string'. * src/cuirass/base.scm (build-event-output-port)[write!]: Use 'bytevector-range' to honor COUNT and OFFSET.
* utils: Add 'bytevector-range'.Ludovic Courtès2018-01-26
| | | | | * src/cuirass/utils.scm (%weak-references): New variable. (bytevector-range): New procedure.
* base: Make repository fetching and compilation non-blocking.Ludovic Courtès2018-01-26
| | | | | | | | Previously these calls would effectively suspend execution of all the fibers until they had completed. * src/cuirass/base.scm (process-specs): Wrap 'fetch-repository' and 'compile' calls in 'non-blocking'.
* utils: Add 'non-blocking' macro.Ludovic Courtès2018-01-26
| | | | | * src/cuirass/utils.scm (%non-blocking): New procedure. (non-blocking): New macro.
* cuirass: Reduce Fiber's tick rate.Ludovic Courtès2018-01-26
| | | | * bin/cuirass.in (main): Pass #:hz to 'run-fibers'.
* cuirass: Catch exceptions in the main fiber and stop everything.Ludovic Courtès2018-01-26
| | | | | | * bin/cuirass.in (main): Add 'exit-channel' and read from it. Catch exceptions in the main fiber and write to that channel upon error.
* logging: Add 'with-time-logging' and use it.Ludovic Courtès2018-01-26
| | | | | | * src/cuirass/logging.scm (call-with-time-logging): New procedure. (with-time-logging): New macro. * src/cuirass/http.scm (handle-builds-request): Use it.
* cuirass: Log pending build request.Ludovic Courtès2018-01-26
| | | | * bin/cuirass.in (main): Print message while fetching pending builds.
* http: Run test within 'run-fibers'.Ludovic Courtès2018-01-26
| | | | * tests/http.scm ("http")["cuirass-run"]: Run server with 'run-fibers'.
* base: Do not restart builds that turn out to have succeeded already.Ludovic Courtès2018-01-26
| | | | | * src/cuirass/base.scm (restart-builds): Mark as succeeded the subset of VALID with at least one valid output.
* base: Do not pass bogus store file names to 'db-update-build-status!'.Ludovic Courtès2018-01-26
| | | | | | * src/cuirass/base.scm (handle-build-event)[valid?]: New procedure. Use it when handling 'build-started', 'build-succeeded', and 'build-failed' events.
* http: Log incoming connections and requests.Ludovic Courtès2018-01-26
| | | | | * src/web/server/fiberized.scm (socket-loop): Add 'log-message' call. * src/cuirass/http.scm (url-handler): Likewise.
* http: Use our own 'fiberized' web server backend.Ludovic Courtès2018-01-25
| | | | | | * src/web/server/fiberized.scm: New file. * Makefile.am (dist_pkgmodule_DATA): Add it. * src/cuirass/http.scm (run-cuirass-server): Use it.
* base: Delete duplicate builds when restarting them.Ludovic Courtès2018-01-25
| | | | | * src/cuirass/base.scm (build-derivation=?): New procedure. (restart-builds): Call 'delete-duplicates' on BUILDS.
* base: Remove unneeded import.Ludovic Courtès2018-01-25
| | | | * src/cuirass/base.scm: Remove unneeded #:use-module.
* http: Sort /api/latestbuilds and /api/queue by time.Ludovic Courtès2018-01-25
| | | | | | | * src/cuirass/database.scm (db-get-builds)[format-order-clause]: Add 'order' clauses. * src/cuirass/http.scm (url-handler): Default to (order finish-time) for /latestbuilds and (order submission-time) for /queue.
* database: Turn 'sqlite-exec' into a macro.Ludovic Courtès2018-01-25
| | | | | * src/cuirass/database.scm (%sqlite-exec): New procedure. (sqlite-exec): New macro.
generated by cgit v1.2.3 (git 2.25.4) at 2024-05-31 23:15:24 +0000