Flash Photography for Data-Driven Hidden Scene Recovery
Matthew Tancik,Guy Satat,Ramesh Raskar
arXiv

Vehicles, search and rescue personnel, and endoscopes use flash lights to locate, identify, and view objects in their surroundings. Here we show the first steps of how all these tasks can be done around corners with consumer cameras. We introduce a method that couples traditional geometric understanding and data-driven techniques. To avoid the limitation of large dataset gathering, we train the data-driven models on rendered samples to computationally recover the hidden scene on real data. The method has three independent operating modes: 1) a regression output to localize a hidden object in 2D, 2) an identification output to identify the object type or pose, and 3) a generative network to reconstruct the hidden scene from a new viewpoint.

Photography Optics in the Time Dimension
Barmak Heshmat,Matthew Tancik,Guy Satat,Ramesh Raskar

Nature Photonics Vol. 13 (2018)

Project WebsiteNatureMIT News

We demonstrate that the imaging optics of an ultrafast camera (or a depth camera) can be dramatically different from the imaging optics of a conventional photography camera. More specifically, we demonstrate that by folding the optical path in time, one can collapse the conventional photography optics into a compact volume or multiplex various functionalities into a single imaging optics piece without losing spatial or temporal resolution. By using time-folding at different regions of the optical path, we achieve an order of magnitude lens tube compression, ultrafast multi-zoom imaging, and ultrafast multi-spectral imaging. Each demonstration was done with a single image acquisition without moving optical components.

Towards Photography Through Realistic Fog
Guy Satat,Matthew Tancik,Ramesh Raskar

ICCP (2018)

Project WebsiteLocal CopyMIT News

Imaging through fog has important applications in industries such as self-driving cars, augmented driving, airplanes, helicopters, drones and trains. Current solutions are based on radar that suffers from poor resolution (due to the long wavelength), or on time gating that suffers from low signal-to-noise ratio. Here we demonstrate a technique that recovers reflectance and depth of a scene obstructed by dense, dynamic, and heterogeneous fog. For practical use cases in self-driving cars, the imaging system is designed in optical reflection mode with minimal footprint and is based on LIDAR hardware. Specifically, we use a single photon avalanche diode (SPAD) camera that time-tags individual detected photons. A probabilistic computational framework is developed to estimate the fog properties from the measurement itself, and distinguish between background pho- tons reflected from the fog and signal photons reflected from the target. 

Object Classification through Scattering Media with Deep Learning on Time Resolved Measurement
Guy Satat,Matthew Tancik,Otkrist Gupta,Barmak Heshmat,Ramesh Raskar

Optics Express Vol. 25, 17466-17479 (2017)

Project WebsiteLocal CopyOSA

A deep learning method for object classification through scattering media.  Traditional techniques to see through scattering media rely on a physical model that has to be precisely calibrated. Computationally overcoming the scattering relies heavily on such physical models, and on the calibration accuracy. Thus, such systems are extremely sensitive to an accurate and lengthy calibration process. Our method trains on synthetic data with variations in calibration parameters that allows the network to learn an invariant model to calibration of lab experiments.

Lensless Imaging with Compressive Ultrafast Sensing
Guy Satat,Matthew Tancik,Ramesh Raskar

IEEE Transactions on Computational Imaging (2017)

Project WebsiteLocal CopyIEEEMIT News

Traditional cameras require a lens and a mega-pixel sensor to capture images. The lens focuses light from the scene onto the sensor. We demonstrate a new imaging method that is lensless and requires only a single pixel for imaging. Compared to previous single pixel cameras our system allows significantly faster and more efficient acquisition. This is achieved by using ultrafast time-resolved measurement with compressive sensing. The time-resolved sensing adds information to the measurement, thus fewer measurements are needed and the acquisition is faster. Lensless and single pixel imaging computationally resolves major constraints in imaging systems design. Notable applications include imaging in challenging parts of the spectrum (like infrared and THz), and in challenging environments where using a lens is problematic.

Gmail Emoji Search

Adding an emoji search box to gmail was one of the projects I worked on during my internship at Google.

Marble Drop
iTunes StoreAndroid Play Store

Drop marbles to solve puzzles in this addictive and challenging puzzle game. Place a variety of colored marbles into funnels so that they arrive in their correct bin. Along the way, watch out for switches, tippers, color mixers and other traps!

Spotify Downloader

Download high quality Spotify songs from their URL. The program reconstructs the PCM frames into a 320 kbps MP3 with metadata. This was not designed to be malicious, simply a personal programing challenge.

SleepTight
Product brochure

SleepTight is designed to improve scoliosis brace usage by increasing patient comfort to encourage consistent use. This is achieved by tightening the brace straps with motors after the patient has fallen asleep. In this project I designed and built the electronics in addition to being the team leader.

Equation Finder
Equation Finder

Can’t quite remember what that equation was? (Work in progress)

Cluck
View in app store

Enjoy fast and fun mobile conversation! Cluck gets your friend's attention like a phone call but lets you text instead of talk.

Timelapse Camera Controller

Motor controlled dolly and camera track controlled by an arduino microcontroller. It can be used to create moving timelapse videos.

Interactive Whiteboard
Github

Work on sketches in real time with others.

Synced IOS Application

Create an interactive shared playlist with your friends and watch as songs, movies, and YouTube videos are played perfectly in sync among all devices.