Teaching Machines to Learn by Metaphors

Teaching Machines to Learn by Metaphors

Teaching Machines to Learn by Metaphors Omer Levy & Shaul Markovitch Technion Israel Institute of Technology Concept Learning by Induction

Few Examples Transfer Learning Target (New)

Source (Original) Define: Related Concept Transfer Learning Approaches Common Inductive Bias

Common Instances Common Features Different Feature Space Example -3 -2

0 2 3 Example -3 -2

0 0 2 3

4 9 Example -3 -2

0 2 3 = 0

4 2 9 Common Inductive Bias

-3 -2 0 0 2 3

4 9 Common Inductive Bias -3 -2

0 0 2 3

4 9 Common Instances -3 -2

0 0 2 3 4

9 Common Features 3 2 4

9 -2 -3 =

2 New Approach to Transfer Learning Our Solution: Metaphors

Metaphors Target (New) Source

(Original) Source Concept Learner

Target Metaphor Learner

h +/-

h ( )= h ( ( ) ) is a perfect metaphor if: 1. is label preserving 2. is distribution preserving

Theorem If is a perfect metaphor - and is a source hypothesis with error - then is a target hypothesis with error

The Metaphor Theorem If is an -perfect metaphor - and is a source hypothesis with error - then is a target hypothesis with error

Redefine Transfer Learning Given source and target datasets, find a target hypothesis such that is as small as possible. Redefine Transfer Learning Given source and target datasets,

find an -perfect metaphor such that is as small as possible. Metaphor Learning Framework Concept Learning Framework

Search Algorithm Hypothesis Space h

Evaluation Function Data Metaphor Learning Framework

Source Search Algorithm Metaphor Space

Evaluation Function Target

Metaphor Evaluation Metaphor Evaluation 1. is label preserving 2. is distribution preserving

Metaphor Evaluation 1. is label preserving Empirical error over target dataset 2. is distribution preserving Statistical distance between and

Metaphor Evaluation ( ( ) , ) Metaphor Evaluation

Metaphor Evaluation ( (

), ) Metaphor Evaluation

Metaphor Spaces Metaphor Spaces General

Few Degrees of Freedom Representation-Specific Bias Geometric Transformations R

Dictionary-Based Metaphors cheese queso Linear Transformations

Which metaphor space should I use? Which metaphor space should I use? Automatic Selection of Metaphor Spaces

Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occams Razor Which metaphor space should I use?

Automatic Selection of Metaphor Spaces Occams Razor Structural Risk Minimization Automatic Selection of Metaphor Spaces

1 2 3 4 Automatic Selection of Metaphor Spaces

1 1 2 2

3 3 4

4 Automatic Selection of Metaphor Spaces 1 1

60 % 2 2

90% 3 3 91 %

4 4 70%

Empirical Evaluation Reference Methods Baseline Target Only Identity Metaphor Merge

State-of-the-Art Frustratingly Easy Domain Adaptation Daum, 2007 MultiTask Learning Caruana, 1997; Silver et al, 2010

TrAdaBoost Dai et al, 2007 Digits: Negative Image Digits: Negative Image

( ) =1 Digits: Negative Image Digits: Higher Resolution Digits: Higher Resolution

Digits: Higher Resolution Wine

Wine Qualitative Results Transfer Learning Target Task Instance

Target Sample Size 1 Digits: Negative Image Digits: Higher Resolution

2 5 10

Discussion Recap Problem: Concept learning with few examples Solution: Metaphors Recap

Problem: Concept learning with few examples Solution: Metaphors Target Source Recap Problem: Concept learning with few examples Solution: Metaphors

Target Source Generic framework Recap Problem: Concept learning with few examples Solution: Metaphors Target Source

Generic framework Wide range of relations Recap Problem: Concept learning with few examples Solution: Metaphors

Target Source Generic framework Wide range of relations

Learn the difference What if the concepts are not related? What if the concepts are not related? Metaphors are not a measure of relatedness

Metaphors are not a measure of relatedness Metaphors explain how concepts are related Vision M ETAPH O R S

Explaining how concepts are related since 2012.

Recently Viewed Presentations

  • Motion Transmission - drrossymathandscience

    Motion Transmission - drrossymathandscience

    Motion transmission systems contain: A driver. component that initiates the motion. At least . ... The shape of the cam determines how the follower will move. ... Slider-crank mechanism. This is the mechanisms used in pistons. Slider-Crank Uses. Trains-wheels.
  • Résultats du CROSS Académique Mercredi 02 Décembre au Pontet

    Résultats du CROSS Académique Mercredi 02 Décembre au Pontet

    Résultats du CROSS AcadémiqueMercredi 02 Décembre au Pontet. BF (course1) MF. NICOLAS Romane 21/208 BERNARD Ciane 16/276. LIOTARD Tiphaine 50/208 LAPERCHE Camille 91/276. HALLUIN Molly 72/208 9/31 par équipe PATRAS Céline 161/276 24/38 par équipe
  • Protostomes - University of California, Irvine

    Protostomes - University of California, Irvine

    Spiral pattern of cleavage in embryos. Clockwise or counter-clockwise leading to different whirls in snails. These are general traits meaning some lophos don't have them. Spiral Cleavage. Ecdysozoan. Grow intermittently by molting (vs. incrementally and continuously)
  • Body Cavities (new page of notes: page 25)

    Body Cavities (new page of notes: page 25)

    Body Cavities. Dorsal cavity protects the nervous system. divided into two subdivisions. Cranial cavity is within the skull and encases the brain. Vertebral cavity runs within the vertebral column and encases the spinal cord
  • Design principles and usability heuristics

    Design principles and usability heuristics

    Be consistent. Provide feedback. Provide clearly marked exits. Provide shortcuts. Deal with errors in a positive manner. Provide help. Heuristic evaluation. Principles can be used to systematically inspect the interface for usability problems
  • Layout-Driven Synthesis for Submicron Technology: Mapping ...

    Layout-Driven Synthesis for Submicron Technology: Mapping ...

    GRASP: A Search Algorithm for Propositional Satisfiability EE878C Homework #2 2002/11/1 KAIST, EECS ICS Lab Lee, Dongsoo Sat in a Nutshell Given a Boolean formula, find a variable assignment such that the formula evaluates to 1, or prove that no...
  • THE HEART Chapter 20 An Introduction to the

    THE HEART Chapter 20 An Introduction to the

    Atrial. Reflex. Also called . Bainbridge. reflex. Adjusts heart rate in response to . venous. return. Stretch receptors in right atrium. Trigger increase in heart rate. Through increased sympathetic activity. 20-4 Cardiodynamics. Hormonal Effects on Heart Rate.
  • ICE-T Project Schedule - NCAR Earth Observing Laboratory

    ICE-T Project Schedule - NCAR Earth Observing Laboratory

    Documentation - 1 April 2011 Rack Layouts (position & weights) Power Requirements AEROS Display Needs Data Channels for NIDAS Data Feeds (ascii, ARINC, etc) Cylinders Upload Schedule Prather* - front position / full rack CVI deMott Anderson DMT- pending WCR...