What happened

Researchers at MIT (a US university), Google (a technology company), and Northeastern University (a US university) introduced InstructMesh, a research tool for repairing AI-generated 3D designs. People can generate an object from text or an image. They then highlight a problem area and describe the fix. They can also use sliders. A preview appears before the system creates a revised model. The work is scheduled for presentation at the ACM Symposium on User Interface Software and Technology, a user-interface research conference, in November 2026.

Why looks are not enough

3D generative models often learn from images. They can copy an object’s appearance without understanding how the object must work. A cup may look like a cup but have a sealed lid. A brace may look stylish but lack strap openings. A model may have walls that are too thin or pieces that do not connect.

The team reconstructed 120 popular Thingiverse designs. Thingiverse is a site where people share 3D-printable models. 78.3% of the reconstructed models had more than one fabrication-related flaw. Each model had 2.4 flaws on average. This is why a model can look convincing on screen, then fail after printing.

How InstructMesh works

The tool does not ask a beginner to edit every part of a complicated mesh. It changes the model’s intermediate latent representation. That is the shape information used between generation and the final surface. InstructMesh builds on TRELLIS, a 3D-generation system from Microsoft, and uses GPT-4, a language model, to connect spoken instructions with repair operations.

A user paints the region needing help. They might ask to open a sealed area, add material, or connect two parts. A preview marks subtractive changes in red and additive changes in green. The user can accept, revise, or cancel the change. The system then decodes the edited representation into a new model.

What the study found

In one user study, 12 people had no previous 3D modeling experience. They identified 90.4% of visible fabrication flaws. An independent expert judged 89.7% of the attempted repairs successful. In a second study, participants compared natural-language instructions with sliders. Language required less effort. Sliders felt slightly more precise. All 12 participants preferred a hybrid workflow that offered both.

These numbers are encouraging, but they are not a guarantee for every object or printer. The tests were small and focused on visible problems.

What remains unresolved

InstructMesh still relies on the person to notice a flaw and select the right area. Some problems appear only during slicing or after a failed print. The study used one main generative model, TRELLIS. The system focuses on static shapes. It does not solve every problem involving hinges or interlocking parts. The printed examples show possible applications, not a complete printability certification.

What to watch next

The researchers want automatic flaw detection and physics simulation. Such checks could ask whether a bowl breaks when dropped or whether a material suits the design. The team also describes possible augmented-reality uses, such as making a phone case that matches nearby objects. Those ideas are future work, not announced products.

The larger lesson is practical. Generative AI does not need to be perfect at the first try. It needs an editing process that keeps people in control. InstructMesh points toward a workflow where AI proposes a shape, a person checks its purpose, and the system repairs a specific weakness before printing.

Sources: MIT News, research paper