| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
Specifically the derivation ordered allocator.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Previously, the allocator worked out the derived priorities for each
build. Unfortunately this is quite a complex query, and took lots of time. As
a result of this, I think the WAL could grow excessively while this long query
was running.
To try and mitigate this, add a new table to keep track of the derived
priorities for unprocessed builds. This requires some maintenance to keep up
to date, which in turn will make things like submitting builds slower, but I
think this might help keep transaction length and WAL size down overall.
|
| |
|
|
|
| |
Mostly so the coordinator for building patches plans more builds at once,
since the allocator is very slow at the moment.
|
| |
|
|
|
| |
As the derivation ordered allocator is quite slow when there's a large number
of builds to consider.
|
| |
|
|
|
| |
This value was unintentionally changed, it's just intended for local
debugging.
|
| |
|
|
| |
As this is preventing some builds from excuting.
|
| | |
|
| | |
|
| |
|
|
| |
As there has been exceptions in this area, the required build priority is #f.
|
| |
|
|
|
| |
Turns out vector-fold and vector-map don't work like I'd expected them to,
like fold and map for vectors.
|
| | |
|
| |
|
|
|
|
|
| |
This translates poorly to JSON, as you can't have multiple values for one name
in a JSON object. Is also is risky in terms of assoc-ref being used, and not
considering more than one key. Using a vector of pairs should help in both
situations.
|
| |
|
|
|
|
|
|
|
| |
If the agent and build tags have overlapping keys, compare the tag value, and
only match the build to the agent if the value matches.
There's a potential issue with some of the code here in that it doesn't cope
well with tags with matching keys but differing values, but this is just a
first implementation.
|
| |
|
|
| |
Just in case this has a bearing on the ordering.
|
| |
|
|
| |
This might help, depending on whether the order they come out of the database.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
In the basic allocator.
|
| |
|
|
|
|
| |
For the derivation ordered allocator. This is a hack to reduce the number of
builds in scope, and I think it's cutting out some builds that I want to
happen.
|
| |
|
|
| |
Changing this to a hash table really speeds things up.
|
| |
|
|
| |
In the basic allocator.
|
| |
|
|
| |
Now that the allocators are faster, such a long backlog isn't helpful.
|
| |
|
|
|
| |
Builds introduced while the allocator is running may cause this bit to crash,
so just ignore any builds that aren't accounted for.
|
| |
|
|
|
| |
Previously, the derivation system was ignored, but this now takes it in to
account. The implementation is copied from the derivation ordered allocator.
|
| |
|
|
|
|
| |
Use a temporary table to avoid computing the priorities for all builds. This
speeds up the allocation to only take a few seconds on the database I'm
testing against.
|
| |
|
|
| |
In the agent and allocator.
|
| |
|
|
|
|
| |
For the derivation ordered allocator. This is an quick alternative for having
some kind of archival mechanism for builds. It should reduce the time it takes
the allocator to run.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
To try and avoid the data from the two queries being inconsistent.
|
| |
|
|
|
| |
This might help avoid any issue with having a unprocessed build, but not
knowing the priority, as it's just been processed.
|
| | |
|
| |
|
|
|
| |
Use EXCEPT, rather than NOT IN to make the SQL query faster. Also, just return
and use the build id, rather than a alist.
|
| |
|
|
|
|
|
| |
It worked under some database conditions, but was very slow under others. Move
more of the logic in to SQL in an attempt to make the allocator faster. This
sort of works, but there were some advantages to the approach before the
approach being replaced in this commit.
|
| | |
|
| |
|
|
|
| |
Previously it was ignoring outputs without builds or build results. This fixes
that.
|
| |
|
|
| |
By using a bigger SQL query, rather than lots of small ones.
|
| |
|
|
| |
Because it's building everything, missing inputs shouldn't be as relevant.
|
| |
|
|
|
| |
This should work better where the build coordinator is being used to build all
derivations in the graph.
|
| |
|
|
|
| |
Previously it was ignoring all setup failures when a single one of the outputs
for a single setup failure should be available.
|
| |
|
|
|
|
| |
As well as sorting by priority, this now looks at which builds are missing
outputs, and which builds will provide those outputs and tries to order the
builds providing the outputs before the builds that need them.
|
| |
|
|
|
|
|
|
|
| |
This should at least do a better job of boosting the priority of builds
required to perform builds which have high priorities.
I'm expecting this to result in contiguous chunks of builds with all the same
priority, and the ordering within those chunks isn't yet decided
intelligently.
|
| |
|
|
| |
As it's not dependent on the agent.
|
| |
|
|
| |
This is called for the same outputs multiple times, so memoize the result.
|
| | |
|
| |
|
|
| |
This will probably perform better when there are lots of step failures.
|