aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* Make fetching metrics work even when having database problemsChristopher Baines2023-07-14
|
* Drop the resource pool timeoutChristopher Baines2023-07-14
| | | | Better to timeout early.
* Reduce the number of threads used by fibersChristopher Baines2023-07-10
| | | | As one thread per core is probably unnecessary.
* Set thread namesChristopher Baines2023-07-10
| | | | To help with debugging
* Expose resource pool statsChristopher Baines2023-07-10
|
* Support getting resource pool statsChristopher Baines2023-07-10
|
* Stop using a pool of threads for database operationsChristopher Baines2023-07-10
| | | | | | Now that squee cooperates with suspendable ports, this is unnecessary. Use a connection pool to still support running queries in parallel using multiple connections.
* Try to fix backfilling blocked_buildsChristopher Baines2023-07-02
|
* Filter out duplicate ids for blocking buildsChristopher Baines2023-07-02
|
* Improve data deletionChristopher Baines2023-07-01
|
* Provide more information about revision processing errorsChristopher Baines2023-06-15
| | | | In the compare package derivations response.
* Fix fetch-narinfo-files to work in the specific outputs caseChristopher Baines2023-06-09
|
* Better report exceptions in query-build-server-substitutesChristopher Baines2023-06-09
|
* Detach the database setup from the main guix-data-service processChristopher Baines2023-06-09
| | | | | This will allow restarting them independently, leaving it up to the operator to ensure that all processes are compatible.
* Query for outputs when build events arriveChristopher Baines2023-06-09
| | | | This will keep the substitute information more up to date.
* Make the build event handling code more genericChristopher Baines2023-06-06
| | | | So that triggering a check for substitutes can be integrated in.
* Extend the invalid parameter handling for more typesChristopher Baines2023-05-22
|
* Have render-compare/package-derivations provide invalid param infoChristopher Baines2023-05-22
| | | | | | So that this can be used by the qa-frontpage. This should be improved and generalised.
* Fix ignoring canceled buildsChristopher Baines2023-05-18
| | | | | The previous changes only affected searching for package derivations, and they also didn't work.
* Ignore canceled builds when querying package derivationsChristopher Baines2023-05-18
| | | | | This will help when using this to submit builds, since you won't end up ignoring derivations with canceled builds.
* Add Guile internal time metricsChristopher Baines2023-05-14
| | | | As this will help identify when the service restarts.
* Ensure the known and unknown keys appearChristopher Baines2023-05-09
|
* Remove redundant match-lambda in select-package-output-availability-for-revisionChristopher Baines2023-05-09
|
* Set lock_timeout for some data deletion transactionsChristopher Baines2023-05-09
| | | | | As these can cause deadlocks. This will probably cause errors, so some retrying will need to be added.
* Use the package_derivations system id in a queryChristopher Baines2023-05-04
| | | | | Rather than the derivations system id, as this helps PostgreSQL run the query faster.
* Further tweak fetching narinfosChristopher Baines2023-04-28
| | | | | Move the batching to the database, which should reduce memory usage while removing the limit on the number of fetched narinfos.
* Tweak fetching narinfosChristopher Baines2023-04-28
| | | | | Currently I'm seeing failures due to guile-gnutls not supporting suspendable ports (write_wait_fd), so batch the requested outputs to try and avoid this.
* Set request timeouts for the thread poolsChristopher Baines2023-04-27
| | | | | The request timeout should ensure that the operations don't back up if the thread pool is overloaded.
* Support request timeouts in the thread poolChristopher Baines2023-04-27
|
* Change par-mapper' to use fetch-result-of-defered-thunksChristopher Baines2023-04-27
| | | | So the fetch-result-of-defered-thunk procedure can be removed.
* Split the thread pool used for database connectionsChristopher Baines2023-04-27
| | | | | | | | In to two thread pools, a default one, and one reserved for essential functionality. There are some pages that use slow queries, so this should help stop those pages block other operations.
* Include distribution counts table in data deletionChristopher Baines2023-04-07
|
* Improve performance of select-fixed-output-package-derivations-in-revisionChristopher Baines2023-03-11
|
* Fix query in get-count-for-next-levelChristopher Baines2023-03-09
|
* Defer backfilling derivation distribution counts until laterChristopher Baines2023-03-09
| | | | After the migrations have run.
* Reorder some vacuum operations when loading revisionsChristopher Baines2023-03-09
|
* Avoid a recursive CTE for finding blocked builds where possibleChristopher Baines2023-03-09
| | | | | | Use the new approach of looking up the distribution of the derivations, and building a non recursive query specifically for this revision. This should avoid PostgreSQL picking a poor plan for performing the query.
* Store the distribution of derivations related to packagesChristopher Baines2023-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This might be generally useful, but I've been looking at it as it offers a way to try and improve query performance when you want to select all the derivations related to the packages for a revision. The data looks like this (for a specified system and target): ┌───────┬───────┐ │ level │ count │ ├───────┼───────┤ │ 15 │ 2 │ │ 14 │ 3 │ │ 13 │ 3 │ │ 12 │ 3 │ │ 11 │ 14 │ │ 10 │ 25 │ │ 9 │ 44 │ │ 8 │ 91 │ │ 7 │ 1084 │ │ 6 │ 311 │ │ 5 │ 432 │ │ 4 │ 515 │ │ 3 │ 548 │ │ 2 │ 2201 │ │ 1 │ 21162 │ │ 0 │ 22310 │ └───────┴───────┘ Level 0 reflects the number of packages. Level 1 is similar as you have all the derivations for the package origins. The remaining levels contain less packages since it's mostly just derivations involved in bootstrapping. When using a recursive CTE to collect all the derivations, PostgreSQL assumes that the each derivation has the same number of inputs, and this leads to a large overestimation of the number of derivations per a revision. This in turn can lead to PostgreSQL picking a slower way of running the query. When it's known how many new derivations you should see at each level, it's possible to inform PostgreSQL this by using LIMIT's at various points in the query. This reassures the query planner that it's not going to be handling lots of rows and helps it make better decisions about how to execute the query.
* Avoid logging when there are no processes to wait forChristopher Baines2023-03-09
|
* Fix counting derivations when there are lotsChristopher Baines2023-03-09
|
* Add an extra index on blocked_buildsChristopher Baines2023-03-05
| | | | I think this might help with queries that don't use the build_server_id.
* Tweak the comparision queryChristopher Baines2023-03-05
| | | | This seems to generate better plans.
* Remove peekChristopher Baines2023-03-05
|
* Set current-guix-package when computing system test derivationsChristopher Baines2023-02-28
| | | | This is a bit ugly, but might speed up computing derivations for system tests.
* Speed up deleting blocked_builds entriesChristopher Baines2023-02-27
|
* Move the delete-unreferenced-derivations advisory lockChristopher Baines2023-02-27
| | | | To better prevent two processes running at the same time.
* Comparing package derivations: Fix inconsistent verbage.Leo Famulari2023-02-17
| | | | | | | | | | Harmonize "Build change" options between the selection menu and the documentation * guix-data-service/web/compare/html.scm (compare/package-derivations): Replace "Still broken" with "Still failing" in the "Build change" help text. Signed-off-by: Christopher Baines <mail@cbaines.net>
* Include some useful numbers on the package derivations comparisonChristopher Baines2023-02-15
| | | | | As it's frequently useful to know how many packages/builds some change has affected.
* Remove redundant postgresql connection when deleting derivationsChristopher Baines2023-02-14
|
* Delete blocked_builds entries when deleting derivationsChristopher Baines2023-02-14
|
generated by cgit v1.2.3 (git 2.25.4) at 2024-05-18 16:26:33 +0000