diff options
| author | Mathieu Othacehe <othacehe@gnu.org> | 2020-10-14 13:49:03 +0200 |
|---|---|---|
| committer | Mathieu Othacehe <othacehe@gnu.org> | 2020-10-14 14:15:09 +0200 |
| commit | b67f38a7b91c8605a3ae9eba1e2bd3da4b579622 (patch) | |
| tree | 4671725a8248dcb66e6a843321a85de715077657 /bin | |
| parent | 514f20a9b53ea575078ab9a413d38646bb48aa0b (diff) | |
| download | cuirass-b67f38a7b91c8605a3ae9eba1e2bd3da4b579622.tar cuirass-b67f38a7b91c8605a3ae9eba1e2bd3da4b579622.tar.gz | |
Queue write operations.
SQLite only allows one concurrent write query operation. Having multiple
database workers calling "db-update-build-status!", will thus increase worker
starvation. Every write operation will also be done is a single transaction.
For those reasons, create a database worker dedicated to write queries. Have
this worker queue work and issue all the queued work queries in a single
transaction.
* .dir-locals.el: Add with-db-writer-worker-thread.
* src/cuirass/database.scm (with-queue-writer-worker): Rename
"with-registration-workers" macro.
(%db-writer-channel): Rename "%db-registration-channel" variable.
(with-queue-writer-worker): Rename "with-registration-workers".
(db-register-builds): Use "with-db-writer-worker-thread" instead of
"with-db-registration-worker-thread".
(db-update-build-status!): Ditto
* src/cuirass/utils.scm (make-worker-thread-channel): Add "queue-size" and
"queue-proc" arguments.
(call-with-worker-thread): Add "options" argument.
* bin/cuirass.in (main): Use "with-queue-writer-worker" instead of
"with-registration-workers". Modify the macro scope to include all the
possible write operations.
Diffstat (limited to 'bin')
| -rw-r--r-- | bin/cuirass.in | 82 |
1 files changed, 40 insertions, 42 deletions
diff --git a/bin/cuirass.in b/bin/cuirass.in index 8da9369..23d8c68 100644 --- a/bin/cuirass.in +++ b/bin/cuirass.in @@ -171,53 +171,51 @@ exec ${GUILE:-@GUILE@} --no-auto-compile -e main -s "$0" "$@" (while #t (log-monitoring-stats) (sleep 600)))))) - (begin + (with-queue-writer-worker + (clear-build-queue) - (clear-build-queue) + ;; If Cuirass was stopped during an evaluation, + ;; abort it. Builds that were not registered + ;; during this evaluation will be registered + ;; during the next evaluation. + (db-abort-pending-evaluations) - ;; If Cuirass was stopped during an evaluation, - ;; abort it. Builds that were not registered - ;; during this evaluation will be registered - ;; during the next evaluation. - (db-abort-pending-evaluations) + ;; First off, restart builds that had not + ;; completed or were not even started on a + ;; previous run. + (spawn-fiber + (essential-task + 'restart-builds exit-channel + (lambda () + (restart-builds)))) - ;; First off, restart builds that had not - ;; completed or were not even started on a - ;; previous run. - (spawn-fiber - (essential-task - 'restart-builds exit-channel - (lambda () - (restart-builds)))) + (spawn-fiber + (essential-task + 'build exit-channel + (lambda () + (while #t + (process-specs (db-get-specifications)) + (log-message + "next evaluation in ~a seconds" interval) + (sleep interval))))) - (spawn-fiber - (essential-task - 'build exit-channel - (lambda () - (with-registration-workers - (while #t - (process-specs (db-get-specifications)) - (log-message - "next evaluation in ~a seconds" interval) - (sleep interval)))))) - - (spawn-fiber - (essential-task - 'metrics exit-channel - (lambda () - (while #t - (with-time-logging - "Metrics update" - (db-update-metrics)) - (sleep 3600))))) + (spawn-fiber + (essential-task + 'metrics exit-channel + (lambda () + (while #t + (with-time-logging + "Metrics update" + (db-update-metrics)) + (sleep 3600))))) - (spawn-fiber - (essential-task - 'monitor exit-channel - (lambda () - (while #t - (log-monitoring-stats) - (sleep 600))))))) + (spawn-fiber + (essential-task + 'monitor exit-channel + (lambda () + (while #t + (log-monitoring-stats) + (sleep 600))))))) (primitive-exit (get-message exit-channel)))))) ;; Most of our code is I/O so preemption doesn't matter much (it |