aboutsummaryrefslogtreecommitdiff
path: root/guix-build-coordinator/datastore
Commit message (Collapse)AuthorAge
* Try to prioritise some database operationsHEADmasterChristopher Baines2024-10-14
| | | | | | At the moment, hooks can get stuck behind new builds being submitted due to the slow writes. Try to manage this by prioritising the write operations from the hooks.
* Revert "Add dbstat metrics"Christopher Baines2024-07-04
| | | | | | It's not fast enough. This reverts commit 09daad84faea3577acf5f1627e4bfed699b10a5d.
* Log the current state of the reader threadsChristopher Baines2024-07-02
|
* Remove now redundant use of the writer thread channelChristopher Baines2024-07-02
| | | | As there are no longer in memory tables.
* Try not using the busy timeoutChristopher Baines2024-07-02
| | | | | I'm not quite sure what effect this will have, but it's probably better to do the waiting in Guile rather than SQLite.
* Add dbstat metricsChristopher Baines2024-07-02
|
* Remove the ATTACH DATABASE lineChristopher Baines2024-06-22
| | | | As it's no longer used.
* Move the allocation plan in to memoryChristopher Baines2024-06-19
| | | | | Rather than using an in memory SQLite table, since I think this might be causing issues when opening new SQLite reader threads.
* Order messages to the writer thread channelChristopher Baines2024-05-15
| | | | So that they're processed in a first come first served manor.
* Tweak running sqitchChristopher Baines2024-04-22
|
* Expose information about setup failuresChristopher Baines2024-02-08
|
* Expose the derived priorities for buildsChristopher Baines2024-01-30
|
* Add more options when updating build prioritiesChristopher Baines2024-01-12
| | | | | | | Computing new derived priorities is slow, so these options allow skipping that if you know what you're doing. For example, if you're updating a batch of related builds and you want them all to have the same priorities and derived priorities, you can now do that much quicker.
* Add delay logging to SQLite transactionsChristopher Baines2023-12-06
|
* Drop the write duration logging thresholdChristopher Baines2023-12-06
|
* Use less database read threadsChristopher Baines2023-08-20
| | | | As I'm not sure that having so many read threads is helpful.
* Switch to a more rigorous method for finding potential source refsChristopher Baines2023-08-10
|
* Ensure references doesn't contain the empty stringChristopher Baines2023-08-10
|
* Change listing builds to work as a streamChristopher Baines2023-05-19
| | | | | | | | | | Both in terms of getting the data from the database, and sending it to the client. This avoids the use of the after-id and ordering by id when listing builds, which makes listing builds faster. It does mean that the database reads may last for a while (which can be a problem), but maybe that can be addressed in other ways.
* Speed up listing buildsChristopher Baines2023-05-17
| | | | | Remove the complicated JOIN on tags and just use subqueries, also remove a redundant join on derivations and don't order the results.
* Remove some left over debuggingChristopher Baines2023-05-11
|
* Fix datastore-list-agent-builds canceledChristopher Baines2023-05-08
|
* Don't error when a build cannot be foundChristopher Baines2023-05-08
|
* Include system uptime in datastore-find-agent-statusChristopher Baines2023-05-06
|
* Include system uptime in the agent status informationChristopher Baines2023-05-05
| | | | As I've found this useful in spotting systems which have problems.
* Name more of the worker thread channelsChristopher Baines2023-05-02
|
* Name the sqlite worker threadsChristopher Baines2023-05-02
| | | | So it's easier to debug issues.
* Don't log as much about "expected" exceptions in worker threadsChristopher Baines2023-05-02
|
* Use #:duration-logger in datastore-call-with-transactionChristopher Baines2023-04-25
| | | | Rather than duplicating what it does.
* Split call-with-time-trackingChristopher Baines2023-04-25
| | | | So that you can observe a duration directly.
* Log long database operationsChristopher Baines2023-04-25
| | | | | There's the delay logger that applies to all operations, but it would be helpful to see any long read or write as well.
* Increase the maximum number of reader threadsChristopher Baines2023-04-24
|
* Be rigerous about always using cached statementsChristopher Baines2023-04-24
| | | | | | The point here isn't really statement caching for performance, but using the cache of statements to keep track of them, and allowing them all to be finalized when the database connection is closed.
* Expire database connection threads on exceptionChristopher Baines2023-04-24
| | | | | | | | | | | I'm concerned that SQLite is very sensitive to prepared statements being left around without being finalized or reset, and I think this can happen if an exception occurs in a worker, but before the reset or finalize happens. Handling this in every individual case would be quite difficult, but given that guile-sqlite finalizes prepared statements when you close a connection, I think that mechanism can be used to somewhat rigerously avoid this (at least for cached prepared statements).
* Add delay logging to more operations involved in submitting buildsChristopher Baines2023-04-24
|
* Log the build-id in datastore-find-build-derivation-systemChristopher Baines2023-04-24
| | | | As I'm not sure why a build wouldn't have a system.
* Reduce the extreme wal size thresholdChristopher Baines2023-04-24
| | | | To avoid the WAL getting too big.
* Make sure the writer thread sets up before the reader threadsChristopher Baines2023-04-24
|
* Use an in memory table for the build_allocation_planChristopher Baines2023-04-23
| | | | Since this doesn't need to be persisted and changes often.
* Enable SQLite URI supportChristopher Baines2023-04-23
|
* Add delay logging for list related derivations with no buildChristopher Baines2023-04-23
|
* Tweak the wal truncation againChristopher Baines2023-04-22
|
* Drop the wal checkpointing attemptsChristopher Baines2023-04-22
| | | | In case this helps with performance.
* Further tune the wal checkpointingChristopher Baines2023-04-22
| | | | | Try to be less disruptive, so allow the wal to grow a bit more, and wait less when the checkpointing is blocked.
* Don't block writes for as long when attempting checkpointsChristopher Baines2023-04-22
|
* Tune the extreme WAL truncatingChristopher Baines2023-04-22
| | | | | Allow the WAL to grow a bit more, and increase the time to wait to truncate it.
* Try much harder to truncate the WALChristopher Baines2023-04-22
| | | | | | I think when there are lots of long reads taking place, that can prevent checkpointing and truncating the WAL, so allow waiting for much longer if this seems to be the case.
* Instrument the closing of writer threadsChristopher Baines2023-04-22
|
* Tweak the busy timeouts for reading and writingChristopher Baines2023-04-22
| | | | | I'm not sure this will necessarily help with the database locking situation, but it's worth a try.
* Don't use exceptions for blocked wal checkpointsChristopher Baines2023-04-22
| | | | As this clutters the output.
generated by cgit v1.2.3 (git 2.45.0) at 2024-11-23 23:12:20 +0000