By subscribing, you agree to our Terms of Use and Policies You may unsubscribe at any time.
Most robots we see today can carry out specific tasks with little human intervention. For example, there exists a robotic arm that can harvest tomatoes. The robot interacts with its environment to perform the task of plucking tomatoes, called robotic manipulation. However, the machine often faces geometrical and physical constraints, such as stability and lack of collision.
To avoid these constraints, researchers at the Massachusetts Institute of Technology (MIT) combined different models, with each addressing a different type of constraint, to develop a new model that can find solutions collectively.
Called the compositional diffusion continuous constraint solver (Diffusion-CCSP), the model learns a family of diffusion models, a kind of generative artificial intelligence models that are trained together, so they share some knowledge, like the geometry of the objects the robot will tackle.
See Also RelatedTheir method uses generative AI to help robots solve transport problems like collisions and the stability of the dumped material. Traditional solutions to these problems are time-consuming.
“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments,” said Zhutian Yang, an electrical engineering and computer science graduate student and lead author the study.
“With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” added Yang.
The researchers’ primary motivation was to solve subproblems that arise during general robot manipulation planning. The researchers explained this in the press release using the example of packing objects in a car. They said one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located. For Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints.
“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” added Yang.
Yang explained that training individual models is time-consuming, costly, and requires a lot of training data. Her team found an alternative approach. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.
“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.
Yang and her team hope to test their model in more complicated situations without needing to be trained on new data.
The study was published in Arxiv.
Study abstract:
This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. Previous methods primarily rely on hand-engineering or learning generators for specific constraint types and then rejecting the value assignments when other constraints are violated. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP) derives global solutions to CCSPs by representing them as factor graphs and combining the energies of diffusion models trained to sample for individual constraint types. Diffusion-CCSP exhibits strong generalization to novel combinations of known constraints, and it can be integrated into a task and motion planner to devise long-horizon plans that include actions with both discrete and continuous parameters.
