The question
VINN (Visual Imitation through Nearest Neighbors) learns visual features with BYOL, then acts by looking up the most similar frames from the demonstrations and averaging their actions. I want to know how its task success changes with the number of demonstrations, and whether the gain comes from the learned features or from the nearest-neighbour way of choosing actions.
Setup
- Tasks: robomimic Lift and Can.
- Demonstrations per task: 10, 50 and 100.
- Methods: VINN, a nearest-neighbour policy on ImageNet features, and behavioural cloning on the same BYOL features as VINN.
Using the same BYOL features for both VINN and behavioural cloning is the point of the design. If they differ, the difference comes from how actions are predicted, not from the features.
Evaluation
Closed-loop rollouts in simulation, starting every method from the same set of initial states, with three independent runs each. I’m recording task success, steps to completion, and the failure patterns that keep coming back.
The project is still in progress, so there are no results on this page yet.