AI turned a Rembrandt masterpiece into 5.6 terabytes of data

What’s better than enjoying an ingenious piece of art in person? Seeing it at 717,000,000,000 pixels on a smartphone or computer screen.

Okay, that might not be completely true—but it’s certainly a distinctive experience, and one that we can thank state-of-the-art technology for. Last week the Rijksmuseum in Amsterdam posted an AI-constructed, ultra-high-res image of “The Night Watch” by Rembrandt. The original piece is nearly 15 feet long and more than 12 feet high and has been under intensive restoration since the early 1900s.

As the story goes, in the 1600s Rembrandt was commissioned by the Amsterdam civic guard to create a sweeping oil painting for their headquarters. The Dutch portraitist constructed a scene with the city’s mayor and his lieutenant—plus 32 other characters, including a dressed-up young lass. The piece is thought to have been completed in 1642, and was moved to the town hall in 1715. Rembrandt was long dead at this point. Without his guidance, the new handlers decided to make a few “edits” to get the painting to their liking: They shaved off a few sections (and subjects) from each of the sides to fit and mount the canvas.

Lucky for art appreciators today, the city commissioned another local painter to draw a smaller reference for the “Night Watch” prior to the hack job. The restoration scientists at Rijksmuseum tapped a form of AI known as neural networks to scale the missing elements from the copy to the original. Another set of algorithms helped them match Rembrandt’s signature light-and-shadow style as they “extended’ the piece back to its full form.

Throughout the two-year-long process, the specialists discovered other secrets hidden in the hefty work. Imaging and mineral scans mapped out traces of calcium, copper, arsenic, and iron, exposing some of the sketches and errors under Rembrandt’s final flourishes. They also noted details that had faded from environmental factors, like smoke.

When the restoration finally wrapped in June of 2021, the team didn’t stop. They swept over the completed painting with a 100-megapixel HD camera, producing 8,439 photos that were then computerized, color-corrected, and stitched together by more algorithms. The laser-sharpened result, shared on the Rijksmuseum website, lets viewers zoom into every .0005-millimeter square at fine resolution. At 717,000,000,000 pixels and 5.6 terabytes, the institution says it’s the largest digital image of an art piece ever.

In a visual sense, this is the closest the public can get to “The Night Watch”—but it allows curators, historians, and neural networks to analyze further aspects of Rembrandt’s work too. Now let’s see if NFT creators will co-opt the technology for their own virtual galleries.