Don't unnecessarily shuffle items before config validation

Before ae693cb7e1 (#9577) we've repeatedly looped over all items in parallel like this:

while not types.done:
  for t in types:
    if not t.done and t.dependencies.done:
      with parallel(all_items, CONCURRENCY) as some_items:
        for i in some_items:
          if i.type is t:
            i.commit()

I.e. all items got distributed over CONCURRENCY threads, but not always equally. E.g. it was the hosts' turn, but only two threads got hosts and did all the work. The others didn't do actual work (due to the lack of hosts in their queue) which reduced the performance. c721c302cd (#6581) fixed it by shuffling all_items first. ae693cb7e1 (#9577) made the latter unnecessary by replacing the above algorithm with this:

while not types.done:
  for t in types:
    if not t.done and t.dependencies.done:
      with parallel(all_items[t], CONCURRENCY) as some_items:
        for i in some_items:
          if i.type is t:
            i.commit()

I.e. parallel() gets only items of type t, so all threads get e.g. hosts.
This commit is contained in:
Alexander Aleksandrovič Klimov 2024-02-19 14:26:06 +01:00 committed by GitHub
parent 04ef105caa
commit 51cdd593da
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 6 deletions

View File

@ -439,12 +439,6 @@ bool ConfigItem::CommitNewItems(const ActivationContext::Ptr& context, WorkQueue
if (!total)
return true;
// Shuffle all items to evenly distribute them over the threads of the workqueue. This increases perfomance
// noticably in environments with lots of objects and available threads.
for (auto& kv : itemsByType) {
std::shuffle(std::begin(kv.second), std::end(kv.second), std::default_random_engine{});
}
#ifdef I2_DEBUG
Log(LogDebug, "configitem")
<< "Committing " << total << " new items.";