Case Study: 3D-360 for education
The New Zealand Virtual Reality Project (NZVR) was started five years ago to give young Kiwis a virtually real experience of their wild places, particularly underwater. In that time the team has delivered a million VR experiences over web, social and in schools.
“To get the real effect of a virtual experience, you have to feel like you’re really there,” says the project’s Director of Photography Richard Robinson. “So most of the shots are by remote... hanging a camera under a drone, propping it up on a tripod, swinging it under a mooring float... anything to get the camera operator out of the picture.”
Students at Kowhai Intermediate have their first experience underwater. NZVR technology allows an educator to play the same video for up to 120 headsets at the same moment.
For his first shot Robinson didn’t even get wet. Instead he connected the camera to the end of a long alloy pole, beneath a polystyrene mooring float, and drifted it through a swarm of penguins feeding in a narrow channel off The Snares, a remote island group south of New Zealand. The penguins were darting all around it, it was terrific material... but the shot was a total disaster. The camera was swinging like a pendulum in the sea-state and the effect in a VR headset was horrendous.
“It was the beginning of a very long journey,” he says.
For the first four years the team was shooting on a 5k monoscopic three-camera unit that was heavy, but practical. They evolved ingenious means of hiding lighting, hiding the operator, and rigging the camera on the bow of boats, under floats and even baited to attract sharks.
Spinner dolphins ride the bow in Niue, South Pacific. The shot was made possible using aluminium rigging through the bow chock.
But something was missing... depth. The team had been using Insta360’s Pro2 and Titan cameras for their terrestrial shoots, both of which were capable of 3D, which allowed a radically new degree of immersion. The possibility of shooting 3D underwater was tantilising, but there were only two cameras in the world—one wasn’t good enough quality, the other was the size of a refrigerator.
The team gathered some partners and began designing the world’s first production 3D-360 housing for the Insta360 Pro2, a reliable, compact workhorse with 8k resolution and a suite of well-designed software for the user, including livestreaming. To maintain a clean zenith and nadir, yet maximise the dome size to meet the focal length of the six lenses, the tolerances were razor thin. There were dozens of prototypes and hundreds of design hours before they settled on the right geometry, which looked for all the world like a cut diamond—six faces to receive the domes, and facets cut to elegantly taper the top and bottom affording an uninterrupted view. It also required a complex parcel of electronics nestled beneath the camera to control it and give feedback to the user—another few hundred design hours. The result was the Mantis Sub, the first commercially available 3D-360 underwater solution for professionals.
Another minor difference with a major impact was the presence of a ISO-limit setting. Rather than hitting maximum ISO in dark environments—which results in a snowstorm of noise in most cameras—the Mantis can be pegged back to a reasonable level (typically 800), allowing smooth shadows. The NZVR crew could film the fragile beauty of black coral forests which grow in near darkness of Fiordland—the first such 360 videos in the world. Not only did the Mantis open up a third dimension, but it opened up the night.
Black coral clings to walls in Dusky Sound, Fiordland, New Zealand. Trees of this size may be a thousand years old, growing in relatively shallow water because of a layer of fresh water a couple of metres thick which paints the surface green and cloaks the depths in darkness. Shooting 360 in this environment had previously been impossible.
Foul weather meant that Robinson’s first dive with the housing was in a public swimming pool. He was immediately impressed with the neutral buoyancy and light weight of the unit and it became the camera of choice for the NZVR Project’s marine sites. He took it from the swimming pool to the subantarctic waters the Auckland Islands to film mating whales.
Zooanthids cloak the bottom of a reef in a narrow channel between islands in Doubtful Sound, Fiordland. The stereoscopic projection makes the viewer reach out to touch the swarm of butterfly perch.
Adding another dimension added new challenges: 3D requires greater stability than monoscopic VR and more even lighting, but the effect is compelling. Fish appear to swim into your hands, and the viewer suddenly has an appreciation of scale and distance.
For schools, the 3D technology allows ’psychological presence’ with the media before, creating more compelling experiences and greater attachment of the learning outcomes.
“It’s next level,” says Robinson. “It was a huge leap forward when we started shooting 3D on land, but there’s something really special about giving that sense of depth underwater. I feel like we’re making underwater experiences real for people for the first time, it really gets you.”