Contrib dist stat #346
Open
Contrib dist stat #346
+190
−39
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This class is equivalent to the previous FedGDO_reset_local and FedGDO_persistent The reset of the local optimizers between each global batch can be set via a mpl_parameter Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Add fedgdo in the mpl.init Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
fix log flake it Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Fix kwargs use for mpl in contributivity S-model initialization could fail if the confusion matrix has not the right shape, which can be the case if some labels are not included in the dataset of a partner. btw I noticed that smodel can only work with datasets with 10 labels, that's only cifar and mnist. I opened an issue about that Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr> Signed-off-by: arthurPignet <arthur.pignet@mines-paristech.fr>
arthurPignet
force-pushed
the
contrib-dist-stat
branch
from
610da4d to
078cbca
Compare
Codecov Report
@@ Coverage Diff @@
## master #346 +/- ##
==========================================
- Coverage 80.68% 80.49% -0.20%
==========================================
Files 15 15
Lines 3045 3128 +83
==========================================
+ Hits 2457 2518 +61
- Misses 588 610 +22
Continue to review full report at Codecov.
|
arthurPignet
requested review from
bowni,
natct10,
RomainGoussault and
Thomas-Galtier
as code owners
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New contributivity measueament based on statistical distances between 2 distributions:
This difference of distributions is interpreted as a noise, which allow us to use a multiheaded adaptation of the smodel method to the multipartner case to estimate and quantify this pseudo-noise.
These contributivity metrics only need inferences to be computed, on the trained model (trained via FedSmodel)
The computational additional cost is thus neglectable
The method doesn't need a 'perfect' and global test dataset.
For now 3 distances are implemented:
These metrics are tested on the reference scenarios, see the colab notebook :
https://colab.research.google.com/drive/1DN1lLdd1b1ZmttmEiQKpx8xW5guEf_f_?usp=sharing
TODO