About Our Solution
DistLearn: A Scalable and Privacy-Preserving Framework for Distributed Federated Learning
6th Semester project by UG3 Students at IIIT Hyderabad
In today's world, data is being generated at an unprecedented scale—across mobile devices, edge servers, and institutions. However, concerns about privacy, security, and high communication costs make it difficult to leverage this data for training machine learning models using conventional centralized techniques. Our project, "DistLearn: Distributed Federated Learning", aims to address this issue by enabling collaborative model training without the need to share raw data.
In federated learning (FL), multiple clients (devices or organizations) train a shared global model using their local data, which never leaves their device. Instead, only model updates are shared with a central server, which aggregates these updates to improve the model iteratively. This ensures that data remains private, while still benefiting from the collective knowledge across clients.
We extend this idea through a Distributed Federated Learning (DFL) framework that tackles key challenges like communication overhead, client selection, and privacy threats.
Some other Key Points:
Privacy & Security Enhancement: By using Encryption, we ensure model updates do not leak sensitive user information.
Custom Client Selection: Not all clients are equally reliable. Our method dynamically selects clients based on their data quality and ability to meet training deadlines, optimizing model performance with fewer communication rounds.
Autonomous Scalability: DistLearn can scale to large numbers of clients while maintaining model quality, making it suitable for real-world applications like healthcare, smart devices, and collaborative platforms.
Technologies & Datasets Used
Development Environment
Our stack leverages Python with PyTorch for deep learning implementations and gRPC for efficient, secure client-server communication.
Training Datasets
MNIST (MLP Multi Class Classification)
ImageHandwritten digit recognition (28×28 grayscale images)
Fashion MNIST (CNN Multi Class Classification)
ImageClassification of 10 clothing categories (28×28 grayscale images)
Diabetes Dataset (MLP Binary Classification)
TabularReal-world medical dataset for predicting diabetes based on health parameters
Timeline
This is the basic timeline of how we went about doing this project over the span of 4 weeks.
Week 1
Model Setup and Baseline Implementation
- Implemented and tested MLP (MNIST, Diabetes) and CNN (Fashion MNIST) models (Centralised).
- Underlying message passing layer using gRPC.
- Created client-side data loaders and model files with preprocessing.
- Implemented FedSGD and FedAvg training logic on client-side.
Week 2
Server-Client Integration and Data Distribution
- Implemented averaging of weights (for FedSGD and FedAvg) on the server.
- Integrated training loop on the client side with architecture reading.
- Developed
setup.pyfor client folder creation and data distribution.
Week 3
Advanced Algorithm Implementation and Optimization
- Added server-side threading to handle multiple clients concurrently.
- Implemented FedAdp and FedCS Algorithm
- Random client selection with fault tolerance for all algorithms.
Week 4
Final Testing, Comparison & Documentation
- Compared algorithms and generated performance graphs.
- Performed client scaling and ablation studies.
- Prepared final project report and presentation.
- Automated multi-client execution (non-terminal).
- Integrated encryption for secure update sharing (noise addition aborted).
Architectures
Architecture diagrams for all the algorithms.
- Overall
- FedSGD
- FedAvg
- FedAdp
- FedCS
Overall Architecture
Ablation Studies
This section highlights our Ablation Studies, where we analyze the impact of different model components on performance. By systematically modifying parts of the model, we identify key factors that influence efficiency and accuracy.
Timing difference between FedSGD and FedAvg
FedSGD vs FedAvg Loss and Accuracy Plots
Effect on training due to change in # of clients
Initial Class Distribution of Dataset
Comparison between FedAvg and FedAdp algo based on skewness of data
Distribution of Client Capabilities for a particular run
Number of clients selected per round in FedCS
Members
Team Members & ContributionContact
Feel free to drop us a message if you have any questions or feedback.Location:
IIIT Hyderabad, Gachibowli, Prof C R Rao Road, Hyderabad, Telangana, 500032
Email:
abhinav.raundhal@students.iiit.ac.in
archisha.panda@research.iiit.ac.in
vinit.mehta@research.iiit.ac.in