Repository navigation
Decentralized controller consistency - #1273
Open
leonard-amsler wants to merge 7 commits into
Open
leonard-amsler wants to merge 7 commits into
leonard-amsler wants to merge 7 commits into
Conversation
…zed-controller-consistency
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
#1227 reworked the federated setting. This PR moves the shared parts to the abstract classes and uses them also in the decentralized setting.
Server
send,crashClient,disconnect, the client ids, theClientConnectedcounter and the connect/disconnect cleanup from the federated controller toTrainingController, so both controllers share them.ClientConnectedClientConnectedClientConnectedgets the same answer again, without counting the peer twice or notifying the other peers.sendhelper, so a failed send disconnects the peer.#syncingNodes.#methods.RetryPeerConnections, and thenConnectionFail, to peers that were fine. There is now one timer perPeersForRound, and the retry count restarts at each round.Client
disconnect()is now shared inClient. It closes the server connection if there is one (local training) and stops waiting for participants, so a client no longer keeps waiting after disconnecting. It also works when called multiple times.sendAndWaitWithRetry.ClientCrashErrorwhen the server crashes it, at the start of a round and while waiting for the server.CrashClientis now part of the decentralized messages.Tests
Fixes #1250