Author(s): Kairos Vance

Originally published on Towards AI.

AI Wrote LaTeX That Compiled Perfectly. The Knot Was Wrong.

I was wandering on Reddit a bit in the last few days. Someone on r/LaTeX posted a photo of a handwritten calculation and asked for help turning it into LaTeX. They had already tried an AI and given up. Nobody had answered.

The calculation is a standard one from a first knot theory course: take the figure-eight knot, switch one crossing, smooth the same crossing, repeat until everything is a circle. The answer at the bottom, 1 − z², is the Conway polynomial of the figure-eight knot. You can check it on the Knot Atlas.

The equations are trivial to typeset. The problem is the six little knot drawings inside the brackets.

I took this request and one easier one, and let an AI deliver both. Along the way it produced four broken figures. Three of them compiled with zero errors. Each of the three looked fine in the code and wrong on the page, and each got caught the same way: render it, look at it.

If you only want the takeaway: an AI-generated TikZ diagram can compile with zero errors and still be wrong, because TikZ errors are about syntax, not about what the picture shows. The only reliable check is to render the PDF and look at it, and for anything with a claim in it (a crossing, a plotted column, a label), to check against a ground truth that lives outside the figure. The rest of this article is the evidence.

The easy one: a pyramid

The second request was a square pyramid with four right triangles highlighted, one per Pythagorean relation: height, slant height, lateral edge, half the base.

There is nothing to report here. The figure is about 30 lines using tikz-3dplot: every point is a 3D coordinate, hidden edges are dashed, each triangle is a filled path through three named points. Everything in this picture is a number, and a model writing code is fine with numbers.

A knot is a different kind of picture.

Why do AI models get knot diagrams wrong in TikZ?

A knot diagram is mostly empty space and a few crossings. All the information is in the crossings: which strand goes over, which goes under. Flip one and you get a different knot, or just a tangled circle.

On a flat page, over and under is drawn as a small gap in the lower strand. So the code has to work out where every crossing lands and then decide, at each one, which strand gets cut. It is a 3D fact squeezed into a 2D drawing, and the model that writes the code never sees the drawing.

The right tool is the knots library from the spath3 package. You draw each strand, it finds the crossings, and an option called flip crossing picks which strand goes on top. With the right tool, it still took four tries.

Failure 1: it compiles, and it’s just circles

The first version drew the curve. Every crossing was two lines touching. Clean compile. Read the code and it looks complete. Look at the PDF and it is a drawing of nothing in particular.

The gaps in the knots library depend on settings (clip width, knot gap) that the first version never set. Nothing in the compiler output says they are missing.

Failure 2: gaps everywhere, wrong knot

This is the dangerous one. There are gaps, it reads as woven, and most people would paste it into a paper.

Follow the strand around. The figure-eight in its simplest drawing alternates: over, under, over, under. This one does not. Two crossings are backwards, so it is not the figure-eight knot. It is a picture of something else that happens to have four crossings.

Looking harder did not fix it. What fixed it was ground truth from outside the picture: the figure-eight is an alternating knot, so going around the strand, every crossing has to flip relative to the one before. Two of the four did not. Flipping those two is one option in the knots library, not a redraw.

(The 3D version of this check also exists: the standard parametrization of the figure-eight has a height coordinate, z = sin 4t, so you can compute which strand is higher at each crossing. Same two crossings come out wrong.)

Failure 3: zero errors, empty brackets

While shortening the code, the four strands got rewritten as a loop (\foreach) instead of four separate lines. Cleaner. Zero errors. The brackets came out empty.

Whatever the loop did to the strands, the compiler did not consider it a problem. The only way to find out was to look at the output. Writing the four strands out by hand brought them back.

Failure 4: the one that complained

The last bug was the only one with an error message: “Dimension too large”, which showed up once the diagrams were scaled down to sit inside the equation. Annoying, but honest.

Final score: four bugs, one error message. The three silent ones were all visual.

The finished calculation

The first line is exact: the figure-eight, the same diagram with one crossing switched (an unknot), and the two-component link from smoothing that crossing (linking number −1). Later lines use simplified pictures, the way the handwritten original does. The answer matches the Knot Atlas.

The whole thing compiles with pdfLaTeX on a stock TeX Live. If you build your own, add draft mode=crossings to any knot to see the crossing numbers, then flip the ones you need.

How do you catch a TikZ figure that compiles but is wrong?

I ran this in HaloMate, where the AI has a computer attached. It wrote the TikZ, compiled it, rendered the PDF to an image, looked at the image, and went back to the code. Every silent failure above was caught at the “looked at the image” step, and the loop ran without me copying anything between windows.

You can do the same loop by hand with any chatbot and Overleaf: paste code, compile, screenshot, paste the screenshot back, describe what is wrong. Same loop. You are the eyes and the clipboard. The Reddit poster had already tried “a different AI” and stopped, which I think is where most people stop: somewhere around round two, when the thing compiles and they assume that means done.

So “which AI is best at LaTeX” is not the useful question. The model that drew circles with no gaps is the same model that fixed it a few minutes later. It did not get smarter. It got to see what it had made.

One more thing, because it is the part I would want someone to tell me. The worst bug here, the wrong knot, was not caught by looking. It looked fine. It was caught by checking each crossing against a fact that lived outside the picture. Rendering catches the figure that is obviously broken. It does not catch the figure that is confidently wrong. For that you need a ground truth, and you need to actually compare against it. Plots that run but show the wrong column. Tables that format nicely around the wrong numbers. Same category.

If the output is a picture, someone has to look at it. If the picture makes a claim, someone has to check the claim. It helps when the AI can do the first part. The second part is still yours.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI