A low-cost in-situ system for continuous multi-person fever screening

It’s human nature to dislike standing in lines and the COVID pandemic has only made queuing even worse, especially in large buildings where it can take hours to have your temperature checked before you are allowed in. Researchers from Columbia Engineering and the Mailman School of Public Health have invented a system that can automatically take temperature readings of people walking by, going about their own business, up to three meters away—no one has to stand in front of a camera for a few seconds to take a measurement. And no one needs to be there to read the measurement and approve the person’s entry.

“By using advanced algorithms and models, we’ve developed an inexpensive way for hospitals, restaurants, subway stations, schools, high-rise buildings with elevators, and more to perform fever screening without disrupting the normal flow of traffic, helping to restore some amount of ‘normal’ to our everyday lives,” said team leader Fred Jiang, associate professor of electrical engineering, who got the idea after seeing how his wife’s hospital was overwhelmed during the pandemic by the numbers of people coming through the entrance. “A nurse had to scan each person in close proximity, and it didn’t guarantee that everyone who passed through did not have fever, as people can take fever reducers.”

Jiang is an expert in intelligent embedded systems and their applications in mobile and wearable computing, intelligent built environments, Internet of Things, and connected health applications. He teamed up with Andrew Rundle, professor of epidemiology, to explore ways to improve the accuracy of inexpensive thermal imagers. There are currently two methods to perform non-contact fever screening: non-contact infrared thermometers like a “thermometer gun” and infrared thermography systems, or “thermal cameras.” These approaches either require people to stand in close proximity of the device for a few seconds, interrupting traffic flow, or are very expensive, preventing wide adoption. Jiang’s expertise in using low-cost sensors combined with signal processing and machine learning algorithms, and Rundle’s broad knowledge of epidemiology led the team to create SIFTER, a low-cost system based on a RGB-thermal camera for continuous fever screening of multiple people.

SIFTER has three components: a sensor node, a cloud server, and a web-based user interface. It uses smaller numbers of low-cost sensors, can run continuously in different environmental conditions, and can screen multiple people simultaneously without requiring active participation from screeners. The system detects and tracks heads in the RGB and thermal domains, constructs thermal heat map models for each tracked person, and classifies people as having or not having fever. It can take key temperature features of heads in-situ at a distance of up to three meters and produce fever screening predictions in real-time, significantly improving screening through-put while minimizing disruption to normal activities.

The team tested SIFTER in the real world, for 10 months in collaboration with ColumbiaDoctors at their midtown clinic in New York City, with support from Teresa Spada, Director of Practice Operations at ColumbiaDoctors Midtown. It screened more than 4,000 people, with a measurement error within 0.4 Fahrenheit at two meters and around 0.6 Fahrenheit at 3.5 meters. In comparison, most infrared thermal scanners on the market, which cost several thousands of dollars, have about a 1 Fahrenheit measurement error measured within 0.5 meters. SIFTER can achieve 100% true positive rate with 22.5% false positive rate without requiring any human interaction, greatly outperforming baseline[6], which sees a false positive rate of 78.5%.

SIFTER’s embedded software is open-source and the hardware costs less than $500 to set up and operate. The data processing component is provided for free by Jiang’s Columbia Intelligent and Connected Systems Lab as a cloud service in containerized environments, allowing anyone to deploy this inexpensive solution in any place around the world.

The entire end-to-end system is fully automated, needing no one to manually take measurements or analyze the results. The researchers built SIFTER using a FLIR One Pro thermal imaging camera along with a Jetson Nano embedded processor to keep their costs low. The system uses signal processing, computer vision, and deep learning to identify relevant “features” or portions of a person’s face, detect the heads of individuals, track their movements, and estimate head orientation using RGB and thermal camera images. Then, based on the head orientation and thermal camera images, the system extracts temperature from patches of skin that are not covered and maps them onto a 3D-point cloud model of everyone’s head. The system takes these temperature readings from different parts of the face and uses machine learning models to estimate the person’s body temperature and determine if he or she has a fever.

To handle the distance people are from the scanner, the researchers first estimated the distance a person’s head is from the camera, based on perspective projection (closer objects appear larger than objects further away) and stereo vision (depth perception based on the distance between RGB and thermal cameras). Then they modeled the physical transmission of heat from a person’s skin to the camera in terms of how far away a person is and used it to calibrate and remove noise resulting from phenomena such as attenuation (the longer a force travels through any medium, the more it is attenuated and the weaker the reading you will receive). Finally, to further improve the system’s efficiency, the team combined physical modeling with data-driven machine learning methods to further improve and account for the drop in temperature readings at distance.

Jiang and Rundle are currently working to improve SIFTER in terms of range and accuracy, so that the system can be deployed in more scenarios, such as in large open spaces and transport hubs. They are also expanding their system to create a “World Wide Flu and Emerging Disease Vision Surveillance System,” which they will jointly run. The team plans to use this system as a public health surveillance tool for detecting emerging infectious diseases and flu, that continuously monitors the number of people out in public while experiencing a fever. When the system detects that the prevalence of people with fever has risen above a low background normal rate, public health authorities will be alerted to launch more in depth infectious disease surveillance and detection protocols.

“We’re very excited about SIFTER,” Rundle added. “Fever is a symptom of many infectious diseases, and this system will provide a low-cost, near-real-time and continuous means of detecting spikes in the number of people experiencing fever. We see SIFTER as a complement to other more costly and slower public health surveillance systems that rely on lab tests and case reporting. We expect this system to help improve detection of emerging infectious diseases and seasonal surges in flu. ”