The fundamental equivalence between deduction and induction states that an inductive learning system’s output can be reproduced by a deductive system if we make the inductive bias explicit.
Formally, if L is a learning algorithm and B is its inductive bias, then for any new instance x’:
This means the classification produced by learning algorithm L on new data x’ after training on dataset D is equivalent to what would be logically deduced from the conjunction of the training data D, the inductive bias B, and the new instance x’.
Where:
- is a learning algorithm
- is a new instance from the instance space
- is the training data set
- is the set of assumptions that constitute the inductive bias of
- represents logical equivalence (“if and only if”)
- represents logical conjunction (“and”)
Example
Induction (Machine Learning)
- You see examples: All swans I’ve seen are white
- You guess a rule: All swans are white
- You predict: The next swan will be white The Fundamental equivalence would then imply the following
- Inductive learning: “I learned from data that swans are white”
- Equivalent deductive system: “IF I assume the data represents all swans (assumption) AND I saw only white swans (data), THEN I can logically conclude all swans are white”