| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Make the number of database reader threads configurable | Christopher Baines | 2024-04-15 |
| | | |||
| * | Make the server parallelism configurable | Christopher Baines | 2024-04-15 |
| | | | | | And drop the default to 1. | ||
| * | Remove + | Christopher Baines | 2024-04-15 |
| | | |||
| * | Fix argument name | Christopher Baines | 2024-03-25 |
| | | |||
| * | Improve handling of cached narinfos when deleting | Christopher Baines | 2024-03-25 |
| | | |||
| * | Rework the cached compressions system | Christopher Baines | 2024-03-25 |
| | | | | | | | | | | | | | | The initial implementation was flawed since guix assumes that any compression mentioned in the narinfo will be available for the lifetime of the narinfo, and the nar-herder was deleting cached compressions without taking this in to account. This commit adds support for scheduling the removal of a cached compression and this schedule is used to inform the TTLs for narinfos. I'm unsure of the value in caching narinfos so maybe some of this complexity can be removed in the future. | ||
| * | Add more TTL options | Christopher Baines | 2024-03-15 |
| | | | | | | | I'm really not sure about this caching stuff, but these options should provide the flexibility to change the TTLs the nar-herder advertises gracefully. | ||
| * | Move most functionality out of the start script | Christopher Baines | 2023-09-12 |
| | | | | | As this can make debugging easier. | ||
| * | Use recent changes to trigger the removal of nars | Christopher Baines | 2023-09-12 |
| | | | | | | Both when the nar has been removed from the database, and to check if it should no longer be stored. | ||
| * | Remove get-nar-files | Christopher Baines | 2023-09-12 |
| | | | | | As it doesn't scale well. | ||
| * | Don't use JIT on aarch64 | Christopher Baines | 2023-09-12 |
| | | | | | | As there's bugs which cause parts of the program to execute more than once https://github.com/wingo/fibers/issues/83 | ||
| * | Name the fibers threads | Christopher Baines | 2023-09-12 |
| | | | | | To help with debugging. | ||
| * | Change the maintenance thread to not be cooperative | Christopher Baines | 2023-09-12 |
| | | | | | As I think this is causing problems. | ||
| * | Make sure to use streaming http requests everywhere | Christopher Baines | 2023-09-12 |
| | | | | | As this avoids blocking. | ||
| * | Add some more logging | Christopher Baines | 2023-09-12 |
| | | |||
| * | Use recent changes to trigger nar mirroring and removal | Christopher Baines | 2023-09-12 |
| | | | | | | As this should be more efficient than repeatedly looking through all the nars in the database. | ||
| * | Use fibers for the recent changes removal thread | Christopher Baines | 2023-09-12 |
| | | |||
| * | Use fibers timeouts for all http requests | Christopher Baines | 2023-09-12 |
| | | |||
| * | Move most maintenance activity to fibers | Christopher Baines | 2023-09-12 |
| | | | | | | On a single separate thread. This will allow for spawning fibers for various maintenance actions. | ||
| * | Switch to cooperative scheduling | Christopher Baines | 2023-09-12 |
| | | |||
| * | Drop the number of fibers threads | Christopher Baines | 2023-07-12 |
| | | |||
| * | Show progress for downloading the database dump | Christopher Baines | 2023-06-30 |
| | | |||
| * | Support setting a storage limit with no nar removal criteria | Christopher Baines | 2023-05-01 |
| | | | | | | | | | As a way of limiting the storage space used, but without removing any nars. While this will mean if you reduce the storage limit, no nars will be removed, I still think this is a useful way to run the nar-herder. | ||
| * | Make import transactional for all the provided narinfos | Christopher Baines | 2023-03-13 |
| | | | | | | As this will allow the --ensure-references-exist option to work properly where there are circular references. | ||
| * | Handle self references when doing checks on narinfo imports | Christopher Baines | 2023-02-15 |
| | | |||
| * | Fix import time reference checking | Christopher Baines | 2023-02-08 |
| | | |||
| * | Allow disabling dumping the database | Christopher Baines | 2023-02-08 |
| | | | | | As this takes up quite a lot of space if the database is large. | ||
| * | Add a flag to allow ensuring the references for a nar are present | Christopher Baines | 2023-02-06 |
| | | |||
| * | Refactor, pulling the narinfo out in to a variable | Christopher Baines | 2023-02-06 |
| | | |||
| * | Don't error when removing files that don't exist | Christopher Baines | 2023-02-06 |
| | | | | | This is useful when removing in bulk and restarting the process. | ||
| * | Add some basic functionality to check the size of the stored files | Christopher Baines | 2023-02-06 |
| | | |||
| * | Make the log-level a base option | Christopher Baines | 2023-02-06 |
| | | |||
| * | Finish an initial implementation of pruning the cached nars | Christopher Baines | 2023-02-04 |
| | | |||
| * | Instrument the cached bytes by compression | Christopher Baines | 2023-02-04 |
| | | |||
| * | Properly implement cached compression levels | Christopher Baines | 2023-02-03 |
| | | | | | It was sort of implemented, but I'd missed some bits. | ||
| * | Be more flexible with cached compressions | Christopher Baines | 2023-02-01 |
| | | | | | | Allow specifying where to fetch the source files from, enabling cached compressions for mirrors. | ||
| * | Make sure to use suspendable ports | Christopher Baines | 2023-02-01 |
| | | | | | This will allow setting timeouts for network activity. | ||
| * | Fix --cached-compression-min-uses handling | Christopher Baines | 2023-02-01 |
| | | |||
| * | Add experimental support for cached compressions | Christopher Baines | 2023-01-16 |
| | | | | | | | This adds optional caching for alternative compressions of stored nars. You could store lzip nars for example, but then compute, cache and provide zstd nars for some stored nars. | ||
| * | Add some instrumentation around the database | Christopher Baines | 2022-06-24 |
| | | |||
| * | Improve handling of nar removal | Christopher Baines | 2022-05-19 |
| | | | | | When --storage is set and not set. | ||
| * | Fiddle more with the nar removal criteria | Christopher Baines | 2022-05-19 |
| | | | | | | This should make the singular stored-on and and'ed stored-on work properly. | ||
| * | Support removing nar files | Christopher Baines | 2022-05-06 |
| | | |||
| * | Initialise the nar_files_total metric when fetching changes | Christopher Baines | 2022-05-04 |
| | | |||
| * | Create the pid file prior to downloading the database | Christopher Baines | 2022-05-04 |
| | | | | | | Otherwise, anything starting the service may time out if the database file is large. | ||
| * | Update the nar_files_total metric when processing changes | Christopher Baines | 2022-05-04 |
| | | |||
| * | Fix the default log-level value | Christopher Baines | 2022-05-04 |
| | | |||
| * | Expose the nar files stats via a new nar_files_total metric | Christopher Baines | 2022-05-04 |
| | | |||
| * | Expose the storage size as a metric | Christopher Baines | 2022-05-04 |
| | | |||
| * | Add support for exposing some basic metrics | Christopher Baines | 2022-04-17 |
| | | | | | In a format understood by Prometheus. | ||