| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Add a fallback TTL for cached compression removals | Christopher Baines | 2024-03-25 |
| | | |||
| * | Improve handling of cached narinfos when deleting | Christopher Baines | 2024-03-25 |
| | | |||
| * | Move all the database operations for cached nars to one fiber | Christopher Baines | 2024-03-25 |
| | | | | | | | The management fiber. This adds the complexity of replying from the management fiber, but hopefully having all the database operations in one place makes things simpler. | ||
| * | Instrument the number of cached compression files | Christopher Baines | 2024-03-25 |
| | | |||
| * | Rework the cached compressions system | Christopher Baines | 2024-03-25 |
| | | | | | | | | | | | | | | The initial implementation was flawed since guix assumes that any compression mentioned in the narinfo will be available for the lifetime of the narinfo, and the nar-herder was deleting cached compressions without taking this in to account. This commit adds support for scheduling the removal of a cached compression and this schedule is used to inform the TTLs for narinfos. I'm unsure of the value in caching narinfos so maybe some of this complexity can be removed in the future. | ||
| * | Close the nar-port rather than the output-port | Christopher Baines | 2024-03-23 |
| | | | | | | As we've finished reading from the nar-port, but the output-port represents the HTTP connection, and is still used. | ||
| * | Update let style | Christopher Baines | 2024-03-23 |
| | | |||
| * | Fix scheduler variable name | Christopher Baines | 2024-03-23 |
| | | |||
| * | Remove with-fibers-timeout | Christopher Baines | 2024-03-23 |
| | | | | | As it's unused. | ||
| * | Rename with-fibers-port-timeouts to with-port-timeouts | Christopher Baines | 2024-03-23 |
| | | | | | As it works with or without fibers. | ||
| * | Tweak the server metrics | Christopher Baines | 2024-03-22 |
| | | | | | | | Remove code in guile-prometheus, and use the new process collector. Also switch to using a non-namespaced registry for these non nar-herder specific collectors. | ||
| * | Spawn cached compression fibers on the maintenance scheduler | Christopher Baines | 2024-03-15 |
| | | | | | So they don't interact with the main scheduler. | ||
| * | Add more TTL options | Christopher Baines | 2024-03-15 |
| | | | | | | | I'm really not sure about this caching stuff, but these options should provide the flexibility to change the TTLs the nar-herder advertises gracefully. | ||
| * | Update metrics while removing nars | Christopher Baines | 2024-03-15 |
| | | | | | Rather than waiting until the end. | ||
| * | Use cooperative fibers for maintenance | Christopher Baines | 2024-03-14 |
| | | |||
| * | Initialise the recent changes count | Christopher Baines | 2024-03-14 |
| | | |||
| * | Stop updating the nar_files_total metric twice | Christopher Baines | 2024-03-14 |
| | | |||
| * | Instrument the recent changes count | Christopher Baines | 2024-03-14 |
| | | |||
| * | Instrument the recent changes limit | Christopher Baines | 2024-03-14 |
| | | | | | This doesn't change, but it's useful to have the value as a metric. | ||
| * | Initialise the storage metrics | Christopher Baines | 2024-03-13 |
| | | | | | This used to happen, and this commit brings it back. | ||
| * | Have guild emit more warnings | Christopher Baines | 2024-02-04 |
| | | |||
| * | Update .envrc to work with the newer direnv | Christopher Baines | 2024-02-04 |
| | | |||
| * | Don't allow the mirror and removal channel to be unspecified | Christopher Baines | 2023-11-23 |
| | | | | | As this is truthy. | ||
| * | Move most functionality out of the start script | Christopher Baines | 2023-09-12 |
| | | | | | As this can make debugging easier. | ||
| * | Make with-fibers-port-timeouts work without fibers | Christopher Baines | 2023-09-12 |
| | | | | | | As it's easy to try and use code in both contexts, so try and support that. | ||
| * | Improve backtraces through call-with-cached-connection | Christopher Baines | 2023-09-12 |
| | | |||
| * | Update retry-on-error procedure from the Guix Build Coordinator | Christopher Baines | 2023-09-12 |
| | | |||
| * | Use recent changes to trigger the removal of nars | Christopher Baines | 2023-09-12 |
| | | | | | | Both when the nar has been removed from the database, and to check if it should no longer be stored. | ||
| * | Improve the metric handling further | Christopher Baines | 2023-09-12 |
| | | | | | | This should avoid the metric being adjusted before it's been set initially. | ||
| * | Remove get-nar-files | Christopher Baines | 2023-09-12 |
| | | | | | As it doesn't scale well. | ||
| * | Use fold-nar-files for check-storage | Christopher Baines | 2023-09-12 |
| | | | | | To avoid calling get-nar-files, as this doesn't scale well. | ||
| * | Don't use JIT on aarch64 | Christopher Baines | 2023-09-12 |
| | | | | | | As there's bugs which cause parts of the program to execute more than once https://github.com/wingo/fibers/issues/83 | ||
| * | Add more logging around requesting recent changes | Christopher Baines | 2023-09-12 |
| | | |||
| * | Cleanup handling of the nar_files_total metric | Christopher Baines | 2023-09-12 |
| | | |||
| * | Remove the get-nar-files call from the fetch changes fiber | Christopher Baines | 2023-09-12 |
| | | | | | As this is quite expensive. | ||
| * | Log the procedure delayed for database operations | Christopher Baines | 2023-09-12 |
| | | |||
| * | Name the fibers threads | Christopher Baines | 2023-09-12 |
| | | | | | To help with debugging. | ||
| * | Change the maintenance thread to not be cooperative | Christopher Baines | 2023-09-12 |
| | | | | | As I think this is causing problems. | ||
| * | Sleep on exceptions in the recent change removal fiber | Christopher Baines | 2023-09-12 |
| | | |||
| * | Show backtraces for fetch-file exceptions | Christopher Baines | 2023-09-12 |
| | | |||
| * | Make sure to use streaming http requests everywhere | Christopher Baines | 2023-09-12 |
| | | | | | As this avoids blocking. | ||
| * | Add some more logging | Christopher Baines | 2023-09-12 |
| | | |||
| * | Enforce an ordering in database-select-recent-changes | Christopher Baines | 2023-09-12 |
| | | |||
| * | Remove some unused imports | Christopher Baines | 2023-09-12 |
| | | |||
| * | Use fibers in fast-download-nars | Christopher Baines | 2023-09-12 |
| | | | | | As it's within a fibers thread now. | ||
| * | Use recent changes to trigger nar mirroring and removal | Christopher Baines | 2023-09-12 |
| | | | | | | As this should be more efficient than repeatedly looking through all the nars in the database. | ||
| * | Use fibers for the recent changes removal thread | Christopher Baines | 2023-09-12 |
| | | |||
| * | Use fibers timeouts for all http requests | Christopher Baines | 2023-09-12 |
| | | |||
| * | Add new fibers timeout utils | Christopher Baines | 2023-09-12 |
| | | | | | | This depends on a tweaked version of (fibers io-wakeup), which calls select of the file descriptors, rather than the ports. | ||
| * | Remove set-socket-timeout | Christopher Baines | 2023-09-12 |
| | | | | | As I don't think it's useful on non blocking sockets. | ||