aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Improve select-job-for-commitChristopher Baines2020-10-09
| | | | It now returns #f or a list, rather than the empty list or a nested list.
* Clarify that the derivations comparison only is for packagesChristopher Baines2020-10-09
| | | | As it excludes the channel instances and system tests.
* Use letpar& for systems and targets in render-compare/derivationsChristopher Baines2020-10-09
|
* Change the derivation comparison targetsChristopher Baines2020-10-09
| | | | To be targets, rather than systems.
* Link to the revisions in the comparison headerChristopher Baines2020-10-09
|
* Guard against errors when recording job stderr outputChristopher Baines2020-10-09
| | | | This might help at least handle errors when inserting data in to PostgreSQL.
* Guard against derivation IDs that aren't numbersChristopher Baines2020-10-09
| | | | | I saw an error suggesting that something came back that wasn't a number, and this should give a more informative error.
* Change table-schema to detect when there are no resultsChristopher Baines2020-10-08
| | | | | | This might give more insight to the [1] errors occuring for some builds. 1: error: couldn't find data for locale in ()
* Just use map rather than par-map& for deleting derivationsChristopher Baines2020-10-08
| | | | As I think par-map& is probably no faster.
* Make the derivation deletion batch size configurableChristopher Baines2020-10-08
|
* Reduce the derivation deletion batch sizeChristopher Baines2020-10-08
|
* Use the git repository fetch with authentication valueChristopher Baines2020-10-07
|
* Support storing whether to authenticate commits by git repositoryChristopher Baines2020-10-07
| | | | To allow using the Guix Data Service when commits aren't authenticated.
* Avoid locking up the thread pool channel on letpar& exceptionsChristopher Baines2020-10-04
| | | | | | | | | | Previously, if an exception occurred during the processing of any but the last letpar& expression, the replies for the other expressions would never be fetched, resulting in that thread in the pool just waiting for a receiver for the message. To avoid this, make sure to read all the replies before raising any exceptions.
* Fix the no latest revision behaviour for the latest revision linksChristopher Baines2020-10-04
| | | | | Previously the render-unknown-revision procedure would error, as it wasn't meant for this purpose.
* Actually delete more of the data for a revisionChristopher Baines2020-10-04
| | | | | | Previously the package_derivations table wasn't considered, which would mean derivations would still be referenced. This commit fixes that, along with also deleting unreferenced entries in some linter related tables.
* Fix delete-unreferenced-derivationsChristopher Baines2020-10-04
|
* Fix the implementation of par-map&Christopher Baines2020-10-04
| | | | It was pretty wrong...
* Remove some non-existent importsChristopher Baines2020-10-04
|
* Extract out the ability to delete a range of commitsChristopher Baines2020-10-04
| | | | | | Some revisions have got disassociated from branches, probably because they were associated with multiple branches in the first place. This should allow deleting them.
* Remove some unnecessary parallel-via-thread-pool-channel callsChristopher Baines2020-10-04
| | | | As these were causing errors because they were nested in letpar&.
* Avoid fiber deadlocksChristopher Baines2020-10-04
| | | | | | | | | | | Channels don't represent some channel on which messages travel, at least not a very long one because it can't accommodate any messages. They simply represent a direct exchange of the message between a sender and receiver. Because of this, put-message blocks the fiber, and if all the threads on the other end are waiting for replies to be received, then you have a deadlock. To avoid this situation, spawn new fibers to send the messages. I think this works at least, although I'm unsure how sensible it is.
* Bump the copyright date in the footerChristopher Baines2020-10-03
| | | | Later is better than never...
* Completely rework the way db connections are handled during requestsChristopher Baines2020-10-03
| | | | | | | | | | | | | Previously, a connection was passed through the code handling the request. When queries were performed, this could block the thread though, potentially leaving the server unable to serve other requests. Instead, this now runs queries in a pool of threads. This should remove the possibility of blocking the threads used by the web server, and in doing so, some of the queries have been parallelised. I''m still not sure about the naming and syntax, but I think the functionality is a sort of step forward.
* Rework the shortlived PostgreSQL specific connection channelChristopher Baines2020-10-03
| | | | | | | In to a generic thing more like (ice-9 futures). Including copying some bits from the (ice-9 threads) module and adapting them to work with this fibers approach, rather than futures. The advantage being that using fibers channels doesn't block the threads being used by fibers, whereas futures would.
* Stop opening a PostgreSQL connection per requestChristopher Baines2020-10-03
| | | | | | | This was good in that it avoided having to deal with long running connections, but it probably takes some time to open the connection, and these changes are a step towards offloading the PostgreSQL queries to other threads, so they don't block the threads for fibers.
* Add some utilities to work with PostgreSQL connections in threadsChristopher Baines2020-10-03
|
* Extract out opening PostgreSQL connectionsChristopher Baines2020-10-03
| | | | So this can be reused.
* Delete derivation_source_files that are unreferencedChristopher Baines2020-10-02
| | | | This will also delete unreferenced derivation_source_file_nars.
* Add an index for derivation_sources.derivation_source_file_idChristopher Baines2020-10-02
| | | | As this speeds up deleting derivation_source_files.
* Make with-postgresql-connection work with multiple valuesChristopher Baines2020-10-02
|
* Change a constraint to add ON DELETE CASCADEChristopher Baines2020-10-02
| | | | | | I've not used these in many places, to try and avoid hiding deleting data, but in this case, this will allow more easily deleting the derivation source file nars, by just deleting the derivation_source_files table entry.
* Reformat lint warning related queryChristopher Baines2020-10-02
|
* Speed up a query for derivation buildsChristopher Baines2020-10-02
| | | | | This change removes a sequential scan from the query plan, making it much faster.
* Change the autovacuum config for some tablesChristopher Baines2020-10-01
| | | | | | Looking at data for the the patches deployment of the Guix Data Service, these tables look like they might benefit from vacuuming/analyzing more often, so adjust the configuration so this will hopefully happen.
* COALESCE a couple more pg_stat fieldsChristopher Baines2020-10-01
| | | | As apparently they can be NULL.
* Track some pg_stat metricsChristopher Baines2020-10-01
| | | | Hopefully this'll help track database things better.
* Drop default thread count for make-postgresql-connection-channelChristopher Baines2020-10-01
| | | | At least for data deletion, 4 seems unnecessary.
* Delete derivations in parallelChristopher Baines2020-10-01
| | | | In an attempt to make this faster.
* Remove the deleting derivations progress outputChristopher Baines2020-10-01
| | | | As this is harder to do when deleting derivations in parallel.
* Delete derivations through a channelChristopher Baines2020-10-01
| | | | Not much different from before, but this will allow parallelising things.
* Add some utilities to use PostgreSQL/Squee through a channelChristopher Baines2020-10-01
| | | | To allow for some concurrency.
* Remove a now redundant part of the maybe-delete-derivation queryChristopher Baines2020-09-30
| | | | As this is covered by the big query selecting the derivation ids.
* Stop recursing now that derivation deletion selection is smarterChristopher Baines2020-09-30
| | | | As this probably won't help with performance.
* Attempt to speed up derivation deletionChristopher Baines2020-09-30
| | | | | | Stop querying for the file-name, as it's unused. Rather than fetching all ids, then looking at each to see if it can be deleted, do some imperfect but not too slow checks in the initial query.
* Remove development code from the process job scriptChristopher Baines2020-09-28
|
* Add a JSON page for repository branchesChristopher Baines2020-09-27
|
* Add a JSON representation for repositoriesChristopher Baines2020-09-27
|
* Fix delete builds for derivation output details setChristopher Baines2020-09-27
|
* Implement the JSON representation for system testsChristopher Baines2020-09-27
|