Proof of Validation
Last updated
Last updated
At the heart of PrompTale AI's federated learning system is the Proof of Validation mechanism, which redefines how data contributions are verified in a decentralized context. This consensus model is designed to manage the challenges posed by non-IID data distributions.
Community-Driven Verification: Multiple users provide feedback on data classifications, establishing a collaborative validation process. This consensus mechanism ensures that no single input disproportionately influences the model’s learning.
Majority Consensus: A majority voting system is employed, where a defined threshold of feedback must align for a classification to be validated.
Weighted Contributions: Feedback from users with established credibility is weighted more heavily in the validation process. This stratification enhances the quality of training data, particularly in contexts influenced by non-IID distributions.
Human Validation: Before the model update, human oversight ensures that the data contributions and classifications are accurate and contextually relevant. This layer of validation adds an additional safeguard, ensuring that only well-validated updates proceed, thus increasing the model’s reliability and trustworthiness.
This is the mathematical representation of PoV
θnew: The candidate model update (new proposed model parameters).
fij: Feedback from generator i on data from user j, where fij=1 if the generator agrees with the update, and 0 if not.
wj: Credibility weight of data source j (user).
N: Total number of data sources (users) contributing data.
T: Threshold for majority consensus.
Elig(θnew): Eligibility check function (1 if the update passes human validation, 0 if it does not).
Breakdown of the PoV Function:
Human Validation Check:
The update is first checked for eligibility through human validation. If it passes (i.e., Elig(θnew)=1), the feedback aggregation continues.
Credibility-Based Weighting of Feedback:
The feedback from each data source j is weighted by the credibility score wj. A more credible user’s feedback will have more influence on the consensus process. The feedback from each generator i is evaluated for every data source j they receive data from.
Majority Consensus:
A majority consensus is applied to the feedback from data sources. If the weighted feedback from users indicates that the majority (threshold T) agrees with the proposed update, it is validated.
Integration into Global Model:
If the update passes both the credibility-based feedback aggregation and human validation, it proceeds to the next phase: integration into the global model.
The Proof of Validation mechanism promotes a transparent environment, enabling users to understand how their contributions influence the model. Users can visualize the direct effects of their input on AI development, thus fostering trust while addressing biases associated with non-IID data.
To protect the model's integrity, PrompTaleAI enforces strict validation protocols for data contributions and model updates with human validation. These mechanisms prevent data tampering, ensuring that the global model resides exclusively on the central server and does not leak onto any client or external device.
