When Reality Catches up with Fiction
Photo credit: Palace of the Governors Photo Archives
[ C H R O N I C L E ]
“Do Detectives Know About These Advances in Image Processing?” While watching your favourite detective serial your blood boils; do they really think that a photo of a barely-distinguishable car, taken on a foggy night is sufficient to be able to make out the number plate? The writes must think the viewers are idiots... or maybe they are aware of the most recent mathematics!
For clarification, let’s take a look at the image processing journal IPOL (for Image Processing OnLine). As well as articles, this journal publishes free and accessible implementations of underlying algorithms. The reader can see some of the extraordinary advances in image processing, some of which are just as incredible, or even more so, than those presented in your detective serial.
Let’s get back to our detectives. Most of the time, they have a photo and from the image, wish to extract anything significant (maliciously called cartoon) from that which is not (called texture). For example, they need to distinguish a fingerprint from the reflections on the surface on which it was found. The quintessence, so to speak, of the image.
For the mathematician, the situation can be expressed in quite similar terms: an image is a function (which associates to each point the characteristics of the image at that point). This function needs to be decomposed into the sum of two sub-functions. Then, all you need to do is define the function that corresponds to the cartoon, what function corresponds to the texture, and define a criterion that allows you to choose, from all the possible decompositions, the one that seems “best”. Once this has been done, a method needs to be shown that can efficiently obtain this optimum.
Research in the field of functional analysis is quite technical and often requires fine intuition of far more complex objects than the images used as an example in this chronicle. The first significant result came in 1992, from the Soviet mathematician (who defected to the USA) Leonid Rudin and his colleagues Stanley Osher and Emad Fatemi - which gave it its nickname of the “ROF method”. The decomposition obtained did not really meet the objectives for use in image processing.
In 2001, Yves Meyer, awarded the prestigious Abel prize in 2017, repeated the ROF method and changed the parameters: a cartoon now needs to belong to a set of functions known as BV (“bounded variations”), a texture has to be within the dual BV space (even more complicated to understand, even for mathematicians) and the selection criterion will be constructed from this choice of properties. This way, he was able to provide a new mathematical proof with a relevant algorithm. Adapted again in 2011 by Antonin Chambolle, from École Polytechnique, and Thomas Pock from Graz Technical University in Austria, this gave rise to the algorithm presented by Vincent Le Guen from Télécom ParisTech to the IPOL journal in 2014, and which you can test online (1).
One question remains: do detectives and screenwriters really know about these leading-edge results? Jean--Michel Morel, in a conference honouring Yves Meyer at ENS Paris-Saclay, showed a few images on which “anonymous” users had applied the algorithm. The details proved beyond a doubt that some had come directly from major American security agencies. Be it good or bad news, it would seem that the FBI is also monitoring progress in mathematics!
> AUTHOR
Roger Mansuy
Mathematician
Roger is a teacher at the Louis-le-Grand secondary school in Paris. Evenry month, he explains a mathematical issue in a public talk.
