| Commit message (Expand) | Author | Age |
|---|
| ... | |
| * | Try setting the en_US.UTF-8 locale, if the normalised fails•••Older versions of the glibc-locales package just contain the UTF-8 variants,
not utf8.
| Christopher Baines | 2019-09-29 |
| * | Don't error when processing revisions of Guix without graft support | Christopher Baines | 2019-09-29 |
| * | Handle loading revisions without the (guix lint) module•••This is a recent addition, so to support loading revisions without it, better
handle the error when loading the module.
| Christopher Baines | 2019-09-29 |
| * | Unset the GUILE_LOAD_* environment variables when using inferiors•••Otherwise modules from outside of the inferior can leak in.
| Christopher Baines | 2019-09-28 |
| * | Add a new table to store package versions by revision ranges•••This isn't new information, but derived from information already in the
database. It's collected here to make querying faster.
The table is updated when each new revision is entered.
| Christopher Baines | 2019-09-27 |
| * | Add newline to the end of the duplicate lint warning output | Christopher Baines | 2019-09-05 |
| * | Improve the package and package-metadata modules•••Add tests around the package module, extract out the use of the
inferior-package record assessors so that they aren't part of the tests, and
switch across the package-metadata module to use
insert-missing-data-and-return-all-ids.
| Christopher Baines | 2019-09-05 |
| * | Add a new table guix_revision_lint_checkers•••To associate a set of lint checkers with a specific revision. While there is
the association through the lint warnings, that only works for checkers where
there are lint warnings for that revision.
Therefore, to allow finding all the checkers for a specific revision, also
associate them directly with the revision.
| Christopher Baines | 2019-09-01 |
| * | Log the manifest derivation when loading new revisions•••As this is useful to know.
| Christopher Baines | 2019-09-01 |
| * | Include glibc-locales from the inferior guix in the environment•••When loading data from an inferior Guix, first build it's latest version of
glibc-locales, and include that in the environment from the inferior.
This improves locale support, which is currently relevant for extracting lint
warnings.
Without this change, you'd only be able to switch to locales supported by the
glibc-utf8-locales package, assuming that the right version is available.
| Christopher Baines | 2019-09-01 |
| * | Log the guix store path being used•••In the load-new-guix-revision module.
| Christopher Baines | 2019-09-01 |
| * | Improve handling of errors when changing locale•••In the load-new-guix-revision module.
| Christopher Baines | 2019-09-01 |
| * | Store lint warnings in the database•••This commit adds the relevant tables and code to store lint warnings in the
database.
Currently, only lint checkers which don't require access to the network will
be run, as this allows the processing to happen without network access. Also,
this functionality won't work in older versions of Guix which don't expose the
lint warnings in a compatible way.
| Christopher Baines | 2019-09-01 |
| * | Add buffering to the log handling for jobs•••To increase the log output speed, avoiding inserting one character at a time
in to the database.
| Christopher Baines | 2019-08-31 |
| * | Avoid erroring when processing emails again•••These changes allow processing emails again, and just creating job and branch
entries where data is missing.
| Christopher Baines | 2019-08-05 |
| * | Clear the previous log entry when starting a job | Christopher Baines | 2019-08-04 |
| * | Display the jobs related to a revision on the revision page | Christopher Baines | 2019-07-22 |
| * | Remove restrictions in select-job-for-update•••This allows running jobs manually that have failed.
| Christopher Baines | 2019-07-19 |
| * | Close the logging port once it's been used | Christopher Baines | 2019-07-19 |
| * | Prioritise processing the latest revisions for branches•••Reserve some capacity to process revisions which are the tip of a branch. This
should reduce the time between new revisions appearing, and then being
processed.
| Christopher Baines | 2019-07-16 |
| * | Record jobs as failed when they are killed•••So that they aren't retried again and again.
| Christopher Baines | 2019-07-14 |
| * | Switch to processing jobs in parallel•••This should speed up processing new revisions, reduce latency between finding
out about new revisions and processing them, as well as help manage memory
usage, by processing each job in a process that then exits.
| Christopher Baines | 2019-07-12 |
| * | Obtain a lock when inserting the data for a new revision•••This is working towards running the jobs in parallel. Each job looks at the
records in the database, and adds missing ones. If other jobs, running in
different transactions insert the same missing records at the same time, this
could cause an error.
Therefore, to just avoid this problem, lock before inserting the data. This
will allow the jobs to be processed in parallel, and it shouldn't have too
much of an effect on performance, as the slow bit is outside of the
transaction.
| Christopher Baines | 2019-07-12 |
| * | Lock around the channel fetch when loading new revisions•••This is in preparation for running jobs in parallel. The channels code in Guix
uses a cached copy of the Git repository. Multiple jobs can't concurrently
access this without causing issues, so use an advisory lock to ensure that
only one job is using the repository at a time.
| Christopher Baines | 2019-07-12 |
| * | Associate a name with database connections•••This helps when working out which connection to the database is doing what.
| Christopher Baines | 2019-07-12 |
| * | Fix using defined? to check for the open-inferior/container function•••It doesn't work as intended unless the module is also specified, so do that.
| Christopher Baines | 2019-07-09 |
| * | Fix issue inserting the empty log row•••If the job started, and then was restarted, the row will already exist. So
don't error on a conflict.
| Christopher Baines | 2019-07-09 |
| * | Move around some of the load new revision code•••To better separate the code that needs to happen after a lock has been
acquired to allow concurrently loading revisions without concurrent insertion
issues.
| Christopher Baines | 2019-07-07 |
| * | Tweak how logs are stored•••Previously, the query for the jobs page was really slow, as it checked the
load_new_guix_revision_job_log_parts table for each job, doing a sequential
scan through the potentially large table.
Adding an index didn't seem to help, as the query planner would belive the
query could return loads of rows, where actually, all that needed checking is
whether a single row existed with a given job_id.
To avoid adding the index to the load_new_guix_revision_job_log_parts table,
and fighting with the query planner, this commit changes the
load_new_guix_revision_job_logs table to include a blank entry for jobs which
are currently being processed. This is inserted at the start of the job, and
then updated at the end to combine and replace all the parts.
This all means that the jobs page should render quickly now.
| Christopher Baines | 2019-07-07 |
| * | Iterate the log viewing•••Replace the Guile-side HTML escaping with a less complete, but hopefully
faster PostgreSQL side HTML escaping approach.
Also, allow reading part of the log, by default, the last 1,000,000
characters, as this should render quickly.
| Christopher Baines | 2019-06-24 |
| * | Direct the build output to the logging-port•••So that this is logged.
| Christopher Baines | 2019-06-22 |
| * | Stop closing the log-port port•••This part of the soft-port seems to be called, but I don't know why, and
trying to close the output port causes issues.
| Christopher Baines | 2019-06-22 |
| * | Record the output from loading new revisions to the database•••So that it can easily be shown through the web interface. There's two tables
being used. One which temporarily stores the output as it's output while the
job is running, and other which stores the whole log once the job has
finished.
| Christopher Baines | 2019-06-22 |
| * | Re-arrange some of the load new revision code•••Try to isolate the code that inserts in to the database, so that the relevant
tables can be locked during this time.
| Christopher Baines | 2019-06-21 |
| * | Add a couple of functions around timing•••That don't rely on timing a single function.
| Christopher Baines | 2019-06-21 |
| * | Log the time loading new revisions takes | Christopher Baines | 2019-06-21 |
| * | Set #:fallback? to #t•••If a substitute fails, just build it anyway.
| Christopher Baines | 2019-06-21 |
| * | Process jobs starting with the most recent•••Conventionally, you'd process the oldest job in a queue, but at the moment, it
would be more useful to have recent information more promptly, and fill in the
historical gaps later. I'm not sure this'll always be the case, but for now,
flip the order in which jobs are processed.
| Christopher Baines | 2019-06-19 |
| * | Use line buffering for the input and output ports•••As these are used for logging, which is done on a line by line basis. Remove
the now redundant calls to (force-output).
| Christopher Baines | 2019-06-17 |
| * | Time everywhere a derivation is built when loading a new revision•••As this is useful information.
| Christopher Baines | 2019-06-17 |
| * | Add a new page listing jobs | Christopher Baines | 2019-06-17 |
| * | Improve the error message when a load revision job fails•••It's useful to know the commit.
| Christopher Baines | 2019-06-15 |
| * | Display a backtrace for errors in channel->derivation-file-name•••To help with debugging.
| Christopher Baines | 2019-06-13 |
| * | Catch the guix available when built•••And use it for the hosting the inferiors, rather than computing the guix
package at runtime. This simplifies the behaviour when the Guix Data Service
is deployed as a Guix package.
| Christopher Baines | 2019-06-13 |
| * | Rework creating /etc when building the derivation for a revision•••Within an inferior. Just use the presence of open-inferior/container directly,
as this is more explicit.
| Christopher Baines | 2019-06-13 |
| * | Add more detailed new revision job handling•••Create a new events table for the new guix revision jobs, and update this when
processing a job starts, as well as finished with success or failure.
Additionally, remove the dependnency on open-inferior/container, as this
functionality isn't merged in to Guix master yet.
| Christopher Baines | 2019-06-02 |
| * | Record job success without deleting the job record•••Previously, the records for jobs would be deleted. It's useful to know when
jobs were inserted in to the database, as well as when they succeeded (if they
have). This change also makes it possible to keep track of jobs that have
failed, as they won't be deleted.
| Christopher Baines | 2019-06-02 |
| * | Store license information for packages•••And display this on the package page.
This uses a couple of new tables, and an additional field in the
package_metadata table.
Currently, the order of the licenses in the package definition isn't stored,
as I'm not sure the order in the list is significant.
| Christopher Baines | 2019-05-15 |
| * | Switch to storing Git repositories in a table•••Rather than just storing the URL in the guix_revisions and
load_new_guix_revision_jobs tables. This will help when storing more
information like tags and branches in the future.
| Christopher Baines | 2019-05-05 |
| * | Try harder to handle errors when loading new revisions•••If the repl enters a bad state, attempt to exit, and catch failures in the
parent process, and rollback the open transaction.
| Christopher Baines | 2019-05-01 |