MIT taught a CAD-generating AI by feeding it its own near-misses

Engineers increasingly use vision-language models to turn 2D design sketches into CAD code that generates 3D models for simulation and testing — a step that normally lets them virtually crash-test or stress-test a design. But existing models often produced CAD programs too simplistic for practical use, and the researchers traced the core bottleneck to a lack of diverse, high-quality CAD training data, not model architecture.

Standard data augmentation creates more training examples by randomly perturbing existing ones — tweaking color, size, shape. The MIT-led team built GIFT instead: a system that generates new training data based specifically on where a given model currently struggles. It samples the model on a CAD-generation problem multiple times in parallel, executes each generated program to check correctness, and separates problems into three buckets — always right, always wrong, and roughly half right.

The 'always right' problems have nothing left to teach the model; GIFT discards them. The 'roughly half right' problems are where GIFT focuses: it corrects the near-miss attempts into working solutions and adds both the corrections and existing successes to a new training set. This process runs through inference-time scaling, so the user picks how much compute to spend without retraining the underlying model from scratch, and the whole loop needs no human correction of the model's mistakes.

Tested against competing data-augmentation approaches, GIFT-trained models produced CAD programs that better matched ground-truth shapes while using only about 20% of the computation. Lead author Giorgio Giannone described the target use case as pointing GIFT at an underperforming CAD model, setting a compute budget, and letting the system convert the model's own errors into its next round of training data.

Generate training data only from the problems a model gets right about half the time

Filtering out problems a model already solves 10/10 times avoids wasting augmentation on examples with no remaining signal to teach.

Filtering out problems it always fails avoids building data around cases with no partial competence to build from — the 50%-success band is where a nudge most reliably converts a near-miss into a win.

Checking correctness by executing the generated CAD code, not scoring by similarity heuristics, gives GIFT a ground-truth signal for which attempts count as near-misses worth correcting.

Running augmentation through inference-time scaling on an already-trained model makes compute cost tunable to the user's budget without retraining the full model.

When a model's bottleneck is generic training data, target new data at its own boundary cases, the problems it gets partly right, since those carry the most learnable signal per example.

The work, led by MIT's DeCoDE Lab in collaboration with the MIT-IBM Computing Research Lab and researchers now at Red Hat and IBM, was presented at the International Conference on Machine Learning. The team's next steps are extending GIFT to also optimize for the manufacturability of the resulting 3D models, not just geometric accuracy, and applying the framework to larger models and a broader range of CAD-generation tasks. The research was funded in part by the MIT-IBM Computing Research Lab.