aboutsummaryrefslogtreecommitdiff
path: root/guix-data-service
Commit message (Collapse)AuthorAge
* Include package details for the package derivation outputs JSONChristopher Baines2020-10-13
| | | | So it's easier to match the output back to the package.
* Tweak linking to build serversChristopher Baines2020-10-11
| | | | | | Move the logic from different places in the view code, and also start supporting linking to guix.cbaines.net builds. I'm unsure quite how to generalise this, but just starting doing it is probably the way forward.
* Improve latest processed revision pages for reproducibility stuffChristopher Baines2020-10-11
| | | | | Support the package-derivation-outputs page, and switch the package-reproducibility page across to use it.
* Add a very rough JSON output for the package derivation outputs pageChristopher Baines2020-10-11
|
* Add another guard clause in to the data deletion codeChristopher Baines2020-10-10
| | | | | | | I've see this error [1] which may relate to the derivation-output-details-id not being a number, so this check should confirm if there is a issue. 1: Throw to key `psql-query-error' with args `(fatal-error "PGRES_FATAL_ERROR" "ERROR: invalid input syntax for integer: \"\"\n")'.
* Split the jobs metric in to succeede, queued and failedChristopher Baines2020-10-09
| | | | Rather than just completed or not.
* Add Prometheus metrics for indexes specificallyChristopher Baines2020-10-09
|
* Include the base commit and target commit in the compare outputChristopher Baines2020-10-09
| | | | This makes it easier to make other requests with the response body.
* Include tablespace as a label for table metricsChristopher Baines2020-10-09
| | | | As this will make it easier to tell where space is going.
* Improve select-job-for-commitChristopher Baines2020-10-09
| | | | It now returns #f or a list, rather than the empty list or a nested list.
* Clarify that the derivations comparison only is for packagesChristopher Baines2020-10-09
| | | | As it excludes the channel instances and system tests.
* Use letpar& for systems and targets in render-compare/derivationsChristopher Baines2020-10-09
|
* Change the derivation comparison targetsChristopher Baines2020-10-09
| | | | To be targets, rather than systems.
* Link to the revisions in the comparison headerChristopher Baines2020-10-09
|
* Guard against errors when recording job stderr outputChristopher Baines2020-10-09
| | | | This might help at least handle errors when inserting data in to PostgreSQL.
* Guard against derivation IDs that aren't numbersChristopher Baines2020-10-09
| | | | | I saw an error suggesting that something came back that wasn't a number, and this should give a more informative error.
* Change table-schema to detect when there are no resultsChristopher Baines2020-10-08
| | | | | | This might give more insight to the [1] errors occuring for some builds. 1: error: couldn't find data for locale in ()
* Just use map rather than par-map& for deleting derivationsChristopher Baines2020-10-08
| | | | As I think par-map& is probably no faster.
* Make the derivation deletion batch size configurableChristopher Baines2020-10-08
|
* Reduce the derivation deletion batch sizeChristopher Baines2020-10-08
|
* Use the git repository fetch with authentication valueChristopher Baines2020-10-07
|
* Support storing whether to authenticate commits by git repositoryChristopher Baines2020-10-07
| | | | To allow using the Guix Data Service when commits aren't authenticated.
* Avoid locking up the thread pool channel on letpar& exceptionsChristopher Baines2020-10-04
| | | | | | | | | | Previously, if an exception occurred during the processing of any but the last letpar& expression, the replies for the other expressions would never be fetched, resulting in that thread in the pool just waiting for a receiver for the message. To avoid this, make sure to read all the replies before raising any exceptions.
* Fix the no latest revision behaviour for the latest revision linksChristopher Baines2020-10-04
| | | | | Previously the render-unknown-revision procedure would error, as it wasn't meant for this purpose.
* Actually delete more of the data for a revisionChristopher Baines2020-10-04
| | | | | | Previously the package_derivations table wasn't considered, which would mean derivations would still be referenced. This commit fixes that, along with also deleting unreferenced entries in some linter related tables.
* Fix delete-unreferenced-derivationsChristopher Baines2020-10-04
|
* Fix the implementation of par-map&Christopher Baines2020-10-04
| | | | It was pretty wrong...
* Remove some non-existent importsChristopher Baines2020-10-04
|
* Extract out the ability to delete a range of commitsChristopher Baines2020-10-04
| | | | | | Some revisions have got disassociated from branches, probably because they were associated with multiple branches in the first place. This should allow deleting them.
* Remove some unnecessary parallel-via-thread-pool-channel callsChristopher Baines2020-10-04
| | | | As these were causing errors because they were nested in letpar&.
* Avoid fiber deadlocksChristopher Baines2020-10-04
| | | | | | | | | | | Channels don't represent some channel on which messages travel, at least not a very long one because it can't accommodate any messages. They simply represent a direct exchange of the message between a sender and receiver. Because of this, put-message blocks the fiber, and if all the threads on the other end are waiting for replies to be received, then you have a deadlock. To avoid this situation, spawn new fibers to send the messages. I think this works at least, although I'm unsure how sensible it is.
* Bump the copyright date in the footerChristopher Baines2020-10-03
| | | | Later is better than never...
* Completely rework the way db connections are handled during requestsChristopher Baines2020-10-03
| | | | | | | | | | | | | Previously, a connection was passed through the code handling the request. When queries were performed, this could block the thread though, potentially leaving the server unable to serve other requests. Instead, this now runs queries in a pool of threads. This should remove the possibility of blocking the threads used by the web server, and in doing so, some of the queries have been parallelised. I''m still not sure about the naming and syntax, but I think the functionality is a sort of step forward.
* Rework the shortlived PostgreSQL specific connection channelChristopher Baines2020-10-03
| | | | | | | In to a generic thing more like (ice-9 futures). Including copying some bits from the (ice-9 threads) module and adapting them to work with this fibers approach, rather than futures. The advantage being that using fibers channels doesn't block the threads being used by fibers, whereas futures would.
* Stop opening a PostgreSQL connection per requestChristopher Baines2020-10-03
| | | | | | | This was good in that it avoided having to deal with long running connections, but it probably takes some time to open the connection, and these changes are a step towards offloading the PostgreSQL queries to other threads, so they don't block the threads for fibers.
* Add some utilities to work with PostgreSQL connections in threadsChristopher Baines2020-10-03
|
* Extract out opening PostgreSQL connectionsChristopher Baines2020-10-03
| | | | So this can be reused.
* Delete derivation_source_files that are unreferencedChristopher Baines2020-10-02
| | | | This will also delete unreferenced derivation_source_file_nars.
* Make with-postgresql-connection work with multiple valuesChristopher Baines2020-10-02
|
* Reformat lint warning related queryChristopher Baines2020-10-02
|
* Speed up a query for derivation buildsChristopher Baines2020-10-02
| | | | | This change removes a sequential scan from the query plan, making it much faster.
* COALESCE a couple more pg_stat fieldsChristopher Baines2020-10-01
| | | | As apparently they can be NULL.
* Track some pg_stat metricsChristopher Baines2020-10-01
| | | | Hopefully this'll help track database things better.
* Drop default thread count for make-postgresql-connection-channelChristopher Baines2020-10-01
| | | | At least for data deletion, 4 seems unnecessary.
* Delete derivations in parallelChristopher Baines2020-10-01
| | | | In an attempt to make this faster.
* Remove the deleting derivations progress outputChristopher Baines2020-10-01
| | | | As this is harder to do when deleting derivations in parallel.
* Delete derivations through a channelChristopher Baines2020-10-01
| | | | Not much different from before, but this will allow parallelising things.
* Add some utilities to use PostgreSQL/Squee through a channelChristopher Baines2020-10-01
| | | | To allow for some concurrency.
* Remove a now redundant part of the maybe-delete-derivation queryChristopher Baines2020-09-30
| | | | As this is covered by the big query selecting the derivation ids.
* Stop recursing now that derivation deletion selection is smarterChristopher Baines2020-09-30
| | | | As this probably won't help with performance.