aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* Remove cross derivation targets that don't make senseChristopher Baines2022-05-27
| | | | This matches the previous behaviour without using the platform data.
* Take advantage of the new (guix platform) moduleChristopher Baines2022-05-26
| | | | | This means there's less reliance on the hardcoded lists of systems and targets and mappings between them.
* Improve the package derivation by guix revision range insert queriesChristopher Baines2022-05-26
|
* Add missing comma in queryChristopher Baines2022-05-24
|
* Fix delete-guix-revision-package-derivation-entriesChristopher Baines2022-05-24
| | | | For the table schema change.
* Fix expected exec-query value when no branch existsChristopher Baines2022-05-24
|
* Fix more issues with the git_commits introductionChristopher Baines2022-05-23
|
* Allow filtering package derivations based on build server buildsChristopher Baines2022-05-23
| | | | | | | | This means you can query for derivations where builds exist or don't exist on a given build server. I think this will come in useful when submitting builds from a Guix Data Service instance.
* Fix more of rebuilding one of the tablesChristopher Baines2022-05-23
|
* Add missing git_branches query part to insert queryChristopher Baines2022-05-23
|
* Raise a clearer exception when a linter crashesChristopher Baines2022-05-23
|
* Partition the package_derivations_by_guix_revision_range tableChristopher Baines2022-05-23
| | | | | | | | | | | | And create a proper git_branches table in the process. I'm hoping this will help with slow deletions from the package_derivations_by_guix_revision_range table in the case where there are lots of branches, since it'll separate the data for one branch from another. These migrations will remove the existing data, so rebuild-package-derivations-table will currently need manually running to regenerate it.
* Fix branch links on the repository pageChristopher Baines2022-05-20
| | | | Thanks to Tobias for reporting.
* Only clear the %store-table when it's definedChristopher Baines2022-05-17
|
* Clear cached store connections when fetching lint warningsChristopher Baines2022-05-13
| | | | | | | | | | | | As I'm seeing the inferior process crash with [1] just after fetching the derivation lint warnings. This change appears to help, although it's probably just a workaround. When there's more packages/derivations, the caches might need clearing while fetching the derivation lint warnings, or this will need to be split across multiple processes. 1: Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
* Clear out cached store connections in the load revision inferiorChristopher Baines2022-05-07
| | | | | These cached store connections have caches associated with them, that take up lots of memory, leading to the inferior crashing. This change seems to help.
* Move the builds.derivation_output_details_set_id updateChristopher Baines2022-04-16
| | | | | | | | | | To the end of the main revision processing transaction. Currently, I think there are issues when this query does update some builds, as those rows in the build table remain locked until the end of the transaction. This then causes build event submission to hang. Moving this part of the revision loading process to the end of the transaction should help to mitigate this.
* Fix a couple of things on the package substitute availability pageChristopher Baines2022-03-12
| | | | | When there's a target, render the heading neatly, and include the target parameter in the URLs.
* Fix an issue with the /builds pageChristopher Baines2022-03-11
|
* Close the load revision inferior prior to inserting dataChristopher Baines2022-03-11
| | | | | This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
* Add to the hardcoded list of valid targetsChristopher Baines2022-03-11
| | | | Since the hardcoded list in the load-new-guix-revision code has been updated.
* Extract out the code for starting an inferiorChristopher Baines2022-03-11
|
* Process each system target pair individuallyChristopher Baines2022-03-11
| | | | As the cross targets take quite some time.
* Disable value history in the inferior replChristopher Baines2022-03-11
| | | | This might help reduce memory usage a little.
* Compute more cross derivationsChristopher Baines2022-03-11
|
* Deduplicate inferior packages including replacementsChristopher Baines2022-03-04
| | | | | | | | | Previously, duplicates could creep through if the duplicate wasn't exported, and only found as a replacement. Now they're filtered out. This isn't ideal, as duplicates aren't always mistakes, it would be useful still to capture this package, but having multiple entries for the same name+version causes the comparison functionality to break.
* Simplify deduplicate-inferior-packagesChristopher Baines2022-03-04
| | | | | Use the a-version and b-version variables, rather than calling the functions again.
* Include the nl_NL.UTF-8 locale when processing revisionsMaxime Devos2022-03-02
| | | | | | | | | | It's 100% translated according to <https://translate.fedoraproject.org/projecs/guix/guix/nl/>. * guix-data-service/model/package-metadata.scm (locales): Add nl_NL.utf-8 Signed-off-by: Christopher Baines <mail@cbaines.net>
* Address a few issues in the load new guix revision testsChristopher Baines2022-03-02
|
* Try to further speed up inserting missing derivation source filesChristopher Baines2022-03-02
| | | | | | | Switch from using a recursive query to doing a breath first search through the graph of derivations, as I think PostgreSQL wasn't doing a great job of planning the recursive queries (it would overestimate the rows involved, and prefer sequential scans for the derivation_outputs table).
* Speed up the finding of missing sourcesChristopher Baines2022-03-01
| | | | Use larger batches and more efficient duplicate deletion.
* Add a faster delete-duplicates functionChristopher Baines2022-03-01
| | | | Which is useful when deleting duplicates from large lists.
* Skip dropping the log part sequence if there's a lockChristopher Baines2022-02-21
| | | | So that the job completes. The sequence can be deleted later.
* Move logging cleanup tasks to after the transaction commitsChristopher Baines2022-02-19
| | | | | | As I think some operations (like the database backup) can block the DROP SEQUENCE bit, so at least this approach means that the main transaction should commit and then the sequence is eventually dropped.
* Change how package supported systems are handledChristopher Baines2022-02-18
| | | | | | This code is a bit tricky, since it should be compatible with old and new guix revisions. I think these changes stop computing package derivations for invalid systems, while hopefully not breaking anything.
* Switch to guile-fibers@1.1Christopher Baines2022-02-09
| | | | | I think the main change required is just to stop accessing the now missing current-fiber parameter.
* Fix some URL encoding issues around derivation namesChristopher Baines2022-01-31
|
* Rename data to nars for package derivation outputsChristopher Baines2022-01-31
| | | | As this is clearer.
* Improve the way the field fields work on various pagesChristopher Baines2022-01-31
| | | | | Remove the brackets from the values since this makes the set of values more consistent, and don't display the no additional fields value on the page.
* Support querying package derivation outputs without the narsChristopher Baines2022-01-31
| | | | Since this speeds up the response if you don't need the nar information.
* Make the revision derivation counts numbersChristopher Baines2022-01-22
|
* Include the commit hash in the revision JSON responseChristopher Baines2022-01-22
| | | | | For the latest processed revision, this is useful for looking up which is the latest processed revision.
* Process derivations in chunksChristopher Baines2022-01-14
| | | | Which should reduce the peak memory usage.
* Fix chunk-for-each! calling proc when there's nothing to doChristopher Baines2022-01-14
|
* Rename chunk-map! to chunk-for-each!Christopher Baines2022-01-14
| | | | As that better reflects what it does.
* Don't count exiting with a status of 0 as a failureChristopher Baines2022-01-14
|
* Re-write insert-derivation-inputs in a more memory efficient manorChristopher Baines2022-01-12
| | | | | | | | | | Previously it would compute a long list of strings, potentially more than 100,000 elements long, then split this string up and insert it in chunks. Only then could memory be freed. This new approach builds the strings in batches for the insertion query, then moves on to the next batch. This should mean that more memory can be freed and reused along the way.
* Record job failure on non-0 exit statusChristopher Baines2022-01-12
| | | | | This is helpful when the jobs fail through Guile running out of memory for example.
* Improve the process jobs script signal handlingChristopher Baines2022-01-09
| | | | | The return value of sleep is unreliable (see guile bug #53139), so use a signal handler instead.
* Increase the default job processing timeoutChristopher Baines2022-01-09
|