Commit message (Collapse) | Author | Age | |
---|---|---|---|
* | Use less database read threads | Christopher Baines | 2023-08-20 |
| | | | | As I'm not sure that having so many read threads is helpful. | ||
* | Switch to a more rigorous method for finding potential source refs | Christopher Baines | 2023-08-10 |
| | |||
* | Ensure references doesn't contain the empty string | Christopher Baines | 2023-08-10 |
| | |||
* | Change listing builds to work as a stream | Christopher Baines | 2023-05-19 |
| | | | | | | | | | | Both in terms of getting the data from the database, and sending it to the client. This avoids the use of the after-id and ordering by id when listing builds, which makes listing builds faster. It does mean that the database reads may last for a while (which can be a problem), but maybe that can be addressed in other ways. | ||
* | Speed up listing builds | Christopher Baines | 2023-05-17 |
| | | | | | Remove the complicated JOIN on tags and just use subqueries, also remove a redundant join on derivations and don't order the results. | ||
* | Remove some left over debugging | Christopher Baines | 2023-05-11 |
| | |||
* | Fix datastore-list-agent-builds canceled | Christopher Baines | 2023-05-08 |
| | |||
* | Don't error when a build cannot be found | Christopher Baines | 2023-05-08 |
| | |||
* | Include system uptime in datastore-find-agent-status | Christopher Baines | 2023-05-06 |
| | |||
* | Include system uptime in the agent status information | Christopher Baines | 2023-05-05 |
| | | | | As I've found this useful in spotting systems which have problems. | ||
* | Name more of the worker thread channels | Christopher Baines | 2023-05-02 |
| | |||
* | Name the sqlite worker threads | Christopher Baines | 2023-05-02 |
| | | | | So it's easier to debug issues. | ||
* | Don't log as much about "expected" exceptions in worker threads | Christopher Baines | 2023-05-02 |
| | |||
* | Use #:duration-logger in datastore-call-with-transaction | Christopher Baines | 2023-04-25 |
| | | | | Rather than duplicating what it does. | ||
* | Split call-with-time-tracking | Christopher Baines | 2023-04-25 |
| | | | | So that you can observe a duration directly. | ||
* | Log long database operations | Christopher Baines | 2023-04-25 |
| | | | | | There's the delay logger that applies to all operations, but it would be helpful to see any long read or write as well. | ||
* | Increase the maximum number of reader threads | Christopher Baines | 2023-04-24 |
| | |||
* | Be rigerous about always using cached statements | Christopher Baines | 2023-04-24 |
| | | | | | | The point here isn't really statement caching for performance, but using the cache of statements to keep track of them, and allowing them all to be finalized when the database connection is closed. | ||
* | Expire database connection threads on exception | Christopher Baines | 2023-04-24 |
| | | | | | | | | | | | I'm concerned that SQLite is very sensitive to prepared statements being left around without being finalized or reset, and I think this can happen if an exception occurs in a worker, but before the reset or finalize happens. Handling this in every individual case would be quite difficult, but given that guile-sqlite finalizes prepared statements when you close a connection, I think that mechanism can be used to somewhat rigerously avoid this (at least for cached prepared statements). | ||
* | Add delay logging to more operations involved in submitting builds | Christopher Baines | 2023-04-24 |
| | |||
* | Log the build-id in datastore-find-build-derivation-system | Christopher Baines | 2023-04-24 |
| | | | | As I'm not sure why a build wouldn't have a system. | ||
* | Reduce the extreme wal size threshold | Christopher Baines | 2023-04-24 |
| | | | | To avoid the WAL getting too big. | ||
* | Make sure the writer thread sets up before the reader threads | Christopher Baines | 2023-04-24 |
| | |||
* | Use an in memory table for the build_allocation_plan | Christopher Baines | 2023-04-23 |
| | | | | Since this doesn't need to be persisted and changes often. | ||
* | Enable SQLite URI support | Christopher Baines | 2023-04-23 |
| | |||
* | Add delay logging for list related derivations with no build | Christopher Baines | 2023-04-23 |
| | |||
* | Tweak the wal truncation again | Christopher Baines | 2023-04-22 |
| | |||
* | Drop the wal checkpointing attempts | Christopher Baines | 2023-04-22 |
| | | | | In case this helps with performance. | ||
* | Further tune the wal checkpointing | Christopher Baines | 2023-04-22 |
| | | | | | Try to be less disruptive, so allow the wal to grow a bit more, and wait less when the checkpointing is blocked. | ||
* | Don't block writes for as long when attempting checkpoints | Christopher Baines | 2023-04-22 |
| | |||
* | Tune the extreme WAL truncating | Christopher Baines | 2023-04-22 |
| | | | | | Allow the WAL to grow a bit more, and increase the time to wait to truncate it. | ||
* | Try much harder to truncate the WAL | Christopher Baines | 2023-04-22 |
| | | | | | | I think when there are lots of long reads taking place, that can prevent checkpointing and truncating the WAL, so allow waiting for much longer if this seems to be the case. | ||
* | Instrument the closing of writer threads | Christopher Baines | 2023-04-22 |
| | |||
* | Tweak the busy timeouts for reading and writing | Christopher Baines | 2023-04-22 |
| | | | | | I'm not sure this will necessarily help with the database locking situation, but it's worth a try. | ||
* | Don't use exceptions for blocked wal checkpoints | Christopher Baines | 2023-04-22 |
| | | | | As this clutters the output. | ||
* | Call sqlite step and reset/finalize together in most places | Christopher Baines | 2023-04-22 |
| | | | | | Since calling reset or finalize is quite important, and this makes the code much simpler. | ||
* | Make datastore-call-with-transaction error messages more useful | Christopher Baines | 2023-04-21 |
| | |||
* | Include whether the build is canceled when listing agent builds | Christopher Baines | 2023-04-21 |
| | | | | | | Canceled builds aren't unallocated from the agent immediately, so including the canceled status in the information given to the agent allows it to not start or potentially stop the build. | ||
* | Add some delay between optimize attempts | Christopher Baines | 2023-04-21 |
| | |||
* | Add some error handling to datastore-find-build-derivation-system | Christopher Baines | 2023-04-21 |
| | | | | Not sure why I'm seeing errors here now, but I am. | ||
* | Add error handling around some date parsing | Christopher Baines | 2023-04-21 |
| | |||
* | Handle COMMIT erroring with busy in datastore-call-with-transaction | Christopher Baines | 2023-04-21 |
| | |||
* | Turns out COMMIT can return busy, so increase the timeout | Christopher Baines | 2023-04-21 |
| | |||
* | Add missing sqlite-step and sqlite-reset | Christopher Baines | 2023-04-20 |
| | |||
* | Improve datastore-call-with-transaction | Christopher Baines | 2023-04-20 |
| | | | | | Include exception in log message, and separate out exception handling for the transaction start and body. | ||
* | Don't truncate the wal from reader threads | Christopher Baines | 2023-04-20 |
| | |||
* | Revert "Retry inside the worker thread" | Christopher Baines | 2023-04-20 |
| | | | | | | As this blocks other database operations that wouldn't wait. This reverts commit 42bf1086d588081548fb1efb0b4cf1e5ad119763. | ||
* | Drop the busy timeout for the write connection | Christopher Baines | 2023-04-20 |
| | | | | Since I don't think it should be busy, so don't wait as long if it is. | ||
* | Instrument datastore-fetch-builds-to-allocate | Christopher Baines | 2023-04-20 |
| | |||
* | Retry inside the worker thread | Christopher Baines | 2023-04-20 |
| | | | | To prevent other operations getting in the way. |