New AI model can generate LEGO designs based on text

A team of researchers from Carnegie Mellon University has unveiled LegoGPT, an AI tool that can create physically stable and buildable LEGO structures based on a textual query. The algorithm generates complete objects with detailed color finishes that can be assembled manually or by robotic manipulators.

GitHub reports.

Here's how the LegoGPT model works

The model is based on the large-scale StableText2Lego dataset, which contains over 47,000 LEGO designs representing more than 28,000 unique 3D objects. Each sample is accompanied by a description created using the GPT-4o model, based on visualizations of the structures from 24 angles.

During the generation process, LegoGPT translates the user's text description into a series of instructions for building the structure, adding one brick at a time. The system verifies the accuracy of each element before adding a new one, and checks for compliance with the brick library and the physical feasibility of placement. If the model detects that a structure is unstable during its creation, all unstable parts are automatically removed, and the process repeats from the last reliable state.

In addition to generating the models themselves, the researchers have implemented a "texturing" mechanism — the visual design of LEGO models according to a given style or color scheme.

This development has already proven effective in creating visually appealing, stable structures that precisely match the original text descriptions. The project code, trained models, and the StableText2Lego dataset have been made publicly available, opening up opportunities for further research in generative design and the educational use of LEGO.

As a reminder, Opera announced a new browser called Neon that is specifically designed for artificial intelligence-based tasks. Currently, access to the product is possible only through pre-registration. After the official launch, the service will operate on a subscription model. The company has not yet disclosed the subscription price.

As we previously reported, Anthropic CEO Dario Amodei stated that modern AI models invent facts less often than humans do. He added that hallucinations in AI are not a significant obstacle to developing AGI (artificial general intelligence) systems.