Concept

Conversations and public understanding of police surveillance are clouded by the fact that surveillance is an abstraction for most. Even as surveillance reaches into all aspects of our lives, our conversations about its impact on our well being are understated. 

Police are one of the most powerful surveillance agents in American society. Under public pressure over recent years, American police have started using body worn cameras (BCWs) and touting them as great transparency tools between police and the public. However, their use of BCWs raises new issues over who controls the images, when and how the cameras should be operated, should the cameras detect faces, etc.

We've used these questions to create an interactive sculpture that questions police power, surveillance, and our ability to confront these powers. Suspended from the ceiling, the humanoid sculpture dons a police tactical uniform draped with four BCWs. Its initial focal point is the realistic 3D printed face, generated with machine learning fed on the faces of American police union leaders. As the audience investigates the face they unwillingly create a threat to the sculpture and it responds. The body cameras open up, revealing themselves to be animatronic eyes that converge on the viewer. Even as you try to escape, the infrared eyes on the body cameras make it known that you can't avoid its gaze once you've been seen. 

The sculpture is the first step in a larger installation about surveillance. By combining new technologies such as facial recognition and robotics with age-old motifs like spiders and puppetry, Asleep with One Eye Open creates visceral feelings and perspectives on surveillance that lets the viewer know that they cannot escape once the eyes have been centered on them.

Fabrication

• The figure is constructed with a half body mannequin, a foam head, stuffed arms and hands, as well as a 3D printed realistic face without eyes.

• It is wearing a vintage SWAT tactical jacket and vest sourced from a military surplus store.

• The figure has a puppet-like pose and is suspended with fishing wire.

• Attached to the PALS vest are four modular sets of 3D printed animatronic eyes with custom housings.

  • The eyes use a pivot system to move along the X & Y axis as well as blink.

  • Each set of eyes are powered by 6 micro-servo motors, which is regulated by a PWM board and powered by an external power supply.

  • All wires are guided through the inside of the jacket and the microcontroller is housed in a large 3D pocket on the back of the vest.

• In the middle of each eyeball and by the fishing wires are lasers, which are made more visible with a haze machine.

Animatronic Eye Design

I was drawn to the realistic and jarring movements of animatronics, especially after encountering this Disney Research paper on realistic robot gazes. I found a CAD model online of a set of animatronic eyes and used Fusion360 to modify it to accommodate the design I had in mind.

CAD model & 3D printing process of the eye components.

Demoing the assembled 3D printed eye mechanism with analog inputs (joystick, button, potentiometer).

To emphasize the directionality of the eyes and add a threatening factor, I added a laser diode in the middle of each eyeball.

Main Body Design

Vintage SWAT tactical uniform & vest purchased from Brooklyn military & police surplus store, stuffed with modular foam to create structure.

The eye case is constructed with leftover cardboard for minimal weight and padded with foam to allow movement of the animatronics. The outside casing was made from black duck canvas to blend in with the tactical uniform and featured strips of fabric that hangs onto the PALS nylon webbing of the vest.

All four sets of eyes attached to the vest with wires threaded through the inside to the Arduino & PWM boards attached to the back with velcro.

Face Design

Since the main feature of the sculpture are the animatronic eyes, I wanted to design a decoy to draw attention from the viewer before the animatronic eyes opened up. People's gazes tend to naturally go towards the face, so I thought it'd be interesting to create a humanoid with a realistic face. I was inspired by Heather Dewey-Hagborg's 3D printed faces in the ITP hallway and decided to use Runway.ML to generate a synthetic face based on former and current chiefs of police unions and departments of the top 5 police per capita cities in the US.

Latent style walk of 250 images trained with 5000 steps via Runway's facial GAN model

Generated synthetic face that looks quite Irish / Italian.

Since we only had a singular low quality image with a fixed view point, I had to exercise some artistic liberties and sculpted the face from scratch in Zbrush.

To simulate human skin, the model was painted in Substance Painter for a realistic look.

Code

The sculpture uses an external web cam with the aid of an infrared light to capture video in low light and is connected to a p5.js sketch running PoseNet & ML5 to detect body parts & positions. It is then connected to the Arduino IDE via serial communication.

Wiring documentation of v1 of the eyes, using an Adruino Uno and Adafruit PCA9685 16-channel servo driver board with an external power supply to drive 6 servo motors. Analog joystick, potentiometer, and button for testing before moving onto serial input.

Face detection success via serial communication from p5.js webcam to Arduino. Utilized pwm libraries and state tracking on the Arduino side to turn on and turn off the servos based on serial signals.

Face tracking success by sending the X & Y coordinates of the face via serial communication and mapping it onto servo motor pulses. Testing a single laser diode in the eyes as well.

To avoid the lasers damaging the viewer’s eyes (they were weak red laser diodes but just to be extra safe) and to capture people wearing masks, we swapped out the original facial recognition library and opted for PoseNet to identify the midpoint between two shoulders.

Code available here.

Networking 4 sets of eyes (24 micro servo motors on 2 chained PWM boards with 2 external power sources). The final configuration required a 5V 10A and 5V 2A supply to power.

Video

Since COVID does not allow for an actual showing of the sculpture, we need to create a compelling video documentation of the form factor and interaction. The challenge is to simultaneously create a dramatic narrative that drives home the concept as well as document the interaction of a viewer.

Storyboards

Credits

A project by Kevin Peter He

In collaboration with Todd Whitney & Daniel Johnston

Sculpture made with Arduino, p5.js, Fusion360, Runway.ML, Zbrush, Substance Painter, 3D printing, and physical fabrications

Video shot on Sony a7iii and edited in Adobe Premiere & After Effects