p2pfl.learning.compression.dp_strategy module

Post-Training Differential Privacy strategy for Local DP.

This implements POST-TRAINING differential privacy, where noise is added once to the final model update, NOT during training (unlike traditional DP-SGD).

DESIGN RATIONALE FOR POST-TRAINING LOCAL DP IN P2PFL:

In a decentralized federated learning environment like p2pfl, we made the critical design decision to implement Local Differential Privacy (LDP) as a “compression” technique rather than a training modification. Here’s why:

  1. NO TRUSTED AGGREGATOR: - In p2pfl, any node can act as an aggregator - There’s no central trusted server - aggregators are just peers - We must assume aggregators could be curious or malicious - Therefore, privacy protection MUST happen before data leaves the client

  2. WHY COMPRESSION FRAMEWORK? - p2pfl converts all models to numpy arrays before transmission - This conversion point is the perfect place to add privacy - Compression pipeline is already framework-agnostic (works with PyTorch, TF, etc.) - No need to modify training code or add framework-specific callbacks

  3. LOCAL DP APPROACH: - Each client adds noise to their own updates (local privacy) - Privacy guarantee holds even if aggregator is malicious - No need for secure aggregation or trusted execution environments - Simple, clean integration with existing p2pfl architecture

  4. IMPLEMENTATION AS “LOSSY COMPRESSION”: - DP is essentially controlled information loss for privacy - Clipping = bounding the information content - Noise = lossy transformation that cannot be reversed - Fits naturally in the compression pipeline alongside quantization, sparsification

This design ensures that privacy protection is: - Automatic: Applied during model encoding, no extra steps needed - Framework-agnostic: Works with any ML framework - Composable: Can combine with other compressions (quantization, TopK, etc.) - Untrusted-aggregator-safe: Privacy guaranteed even against malicious aggregators

Refs:

  1. “Deep Learning with Differential Privacy” (Abadi et al., 2016) https://arxiv.org/abs/1607.00133 - Introduces DP-SGD with gradient clipping and noise addition - Foundation for our clipping + Gaussian noise approach

  2. “cpSGD: Communication-efficient and differentially-private distributed SGD” (Agarwal et al., 2018) https://arxiv.org/abs/1805.10559 - Combines compression with differential privacy - Shows DP can be viewed as a form of lossy compression

  3. “Differentially Private Federated Learning: A Client Level Perspective” (Geyer et al., 2017) https://arxiv.org/abs/1712.07557 - Focuses on client-level DP in federated settings - Emphasizes importance of local noise addition for untrusted aggregators

  4. “Local Differential Privacy for Federated Learning” (Truex et al., 2020) https://arxiv.org/abs/1911.04882 - Specifically addresses LDP in federated learning with untrusted servers - Supports our approach of applying DP before transmission

class p2pfl.learning.compression.dp_strategy.DifferentialPrivacyCompressor[source]

Bases: TensorCompressor

Apply POST-TRAINING local differential privacy as a compression technique.

This implements POST-TRAINING DP, where privacy is applied once to the final model update after training completes, NOT during training iterations.

Key differences from traditional DP-SGD: - Traditional DP-SGD: Clips/adds noise to gradients at EACH training step - Post-Training DP: Clips/adds noise ONCE to the final model update

This approach: 1. Requires NO modifications to local training code 2. Works with any ML framework 3. Typically has less accuracy loss (noise added once vs. many times)

Note: The params argument is expected to be the model update (delta) after local training, not the full set of model weights.

apply_strategy(params, clip_norm=1.0, epsilon=3.0, delta=1e-05, noise_type='gaussian', stability_constant=1e-06)[source]

Apply differential privacy to model parameters.

Parameters:

  • params (list[ndarray]) – Model update (delta) after local training

  • clip_norm (float) – Maximum L2 norm for clipping (C)

  • epsilon (float) – The privacy budget (ε). Lower values correspond to stronger privacy guarantees. Represents the maximum “information leakage” allowed for a single data point. A typical value is between 0.1 and 10.0.

  • delta (float) – The privacy budget (δ), typically a small number (e.g., 1e-5). It represents the probability that the privacy guarantee of epsilon is broken. It should be less than 1/N, where N is the number of data points in the dataset. This parameter is only used for Gaussian noise.

  • noise_type (str) – Type of noise to add (“gaussian” or “laplace”)

  • stability_constant (float) – A small constant to avoid division by zero when clipping.

Return type:

tuple[list[ndarray], dict]

Returns:

Tuple of (dp_params, dp_info) where dp_info contains privacy parameters

reverse_strategy(params, additional_info)[source]

Reverse the differential privacy transformation.

Note: DP is inherently irreversible (that’s the point!), so this just returns the parameters as-is. The additional_info contains metadata about what DP was applied.

Parameters:

  • params (list[ndarray]) – The DP-protected parameters

  • additional_info (dict) – Contains DP metadata (clip_norm, noise_multiplier, etc.)

Return type:

list[ndarray]

Returns:

The same parameters (DP cannot be reversed)

p2pfl.learning.compression.dp_strategy.LocalDPCompressor

alias of DifferentialPrivacyCompressor