Object Behavior Test Bed

Set up an environment to test modeling and inferring object behavior under various conditions.

This item relates to the broader goal of modeling object behaviors in Monty.

We can test Monty's ability to model object behaviors with successively more difficult tasks:

Level 1:

• Objects that move repeatedly in the same way

Level 2:

• Different morphologies with the same behavior
• Same behavior at varying orientations on morphologies
• Objects that stop moving at some points in the sequence

Level 3:

• One object has multiple behaviors at different locations

Level 4:

• Same behavior at varying speeds

Level 5:

• Behaviors that involve a change in features instead of movement

Initial Experiment Setup

• 1 sensor patch that gets processed by 2 SMs (one for static and one for moving features) and sent to 2 LMs.
• Supervision during learning -> provide object ID, pose & state in sequence at each step.
• Evaluate accuracy & speed of recognizing behavior & morphology ID, pose & state (+visualize).
• Could start with 2D environment to make the state dimension easier to visualize (keep in mind potential difficulty in extracting movement from a 3D depth image).

Existing Test Environment

An early version of a simple object behavior environment can be found here. It can be used to simulate physical joints and springs and retrieve locations and orientations in CMP format. See a simple example of a "stapler"/hinge-object below.

Simplified simulated stapler

---

Help Us Make This Page Better

All our docs are open-source. If something is wrong or unclear, submit a PR to fix it!

Make a Contribution

Learn how to contribute to our docs