Autodesk threw a fair and invited a few other software vendors to celebrate the emergence of capture–compute–create workflows. 

By Kathleen Maher

In a burst of hey-kids-let’s-put-on-a-show, Autodesk kicked off its Real 2015 conference on a beautiful Feb­ruary day at San Francisco’s Fort Mason. The site, a reclaimed military base, is a longtime center for the city’s experimenters, artists, and young entrepreneurs, and that’s how Autodesk wants to see itself as it builds new busi­ness it sees growing in the spaces be­tween the computer graphics capabili­ties it has built and acquired and the op­portunities to connect that technology with the real world. The conference is a way for Autodesk to make friends with the people in the forefront of change, and some people joked that it’s also a way for Autodesk to spot new acquisition targets.

Autodesk’s Carl Bass has a vision for Autodesk, and it’s based on what he likes to do in his garage—make stuff. In one sense he’s banking on what all boomers bank on: If there’s one of me who likes this, there’s millions of me’s who like this. But Autodesk is also very tuned in to what’s going on out on the horizon. This is a company that has been in the crosshairs of many a com­petitor for a long time, and Autodesk has had its down times, but through it all they have managed to survive and maintain a major market share.

One of the ways the company has done this is to be nimble. They might not be first, but the company tries to be in a position to adapt quickly. These days are different; there really seem to be some shifts coming in the way busi­ness and fun get done, and Autodesk knows this. Bass knows this from work­ing in his workshop. The rest of Autodesk knows this because the company has made it a habit to acquire smart companies and smart people.

What’s going to happen is the way we design and make products, the way we will use the world around us, and the way we develop software and entertainment are going to change. We’ll be able to use everything that’s around us like building blocks to build new stuff however we want in the real world or in the digital world. Someday, maybe not too long from now, it will seem as strange to try to design a prod­uct using the clunky CAD tools we have as it would be to try to communicate by making signs in the sand with sticks.

Real 2015 was organized around three activities: capture, compute, cre­ate. The speakers were from companies that are using new technologies to per­form reality capture using cameras and scanners and sensors. They made sense of the data collected using computers, and then they made something new out of it all. There were also talks from art­ists, designers, and creative people who are looking for new ways to share ideas with people.

AR is real

Naturally, augmented reality (AR) in all its myriad forms was a major theme at the show. Com­panies are coming out of the woodwork after spending a few years in their R&D garages working with the PrimeSense camera and SDK. PrimeSense, as you’ll remember, was the Israeli company that built a 3D camera that was used by Microsoft to build the Kinect. The com­pany was acquired by Apple, and all has gone quiet on that front but not before thousands of developers got hold of the PrimeSense SDK and figured out that there’s just a whole lot of cool stuff you can do if you can capture the world around you using a cheap little 3D camera and a tablet. The gauntlet has been picked up by Google for the Proj­ect Tango camera for Android devices. Mantis Vision has partnered with the Project Tango team to build the Aquila tablet with Mantis’ MV4D structured light camera technology.

Intel has developed the RealSense camera (working with Creative and their Senz3D technology). It’s now in PCs and coming soon to more devices. Google, Intel, and Apple can all deliv­er millions of users with a mere prod­uct launch, but for this current crop of apps, the PrimeSense SDK has provided the baseline development platform for most of them.

Companies like Matterport and Para­cosm are building 3D mapping technol­ogy that enables people to quickly build 3D maps of places. So, then what? A 3D map of an interior space means that ro­bots can traverse rooms safely. It means firefighters might be able to navigate smoke-filled corridors, and most of all, it means your house can be turned into an awesome 3D world where you can play games. Build your world and then play in it. Pair this technology with VR headsets and it’s possible to capture real estate property and offer people walk-throughs of homes for sale without even disturbing the present inhabitants. Real estate agents say that when people come to look at a house in person after they’ve walked through it virtually, it’s much easier to close the deal. Those people already know they really like the house even before they walk through the front door.

Floored

Floored, a startup based in New York and San Francisco, is developing tools that take the 2D floor plans you see on Zillow’s StreetEasy site and turn them into 3D spaces that potential buy­ers can use to place furniture and otherwise fantasize about their new home. Zillow’s ambitions are relatively mod­est, with simple 3D models, but just the ability to quickly turn 2D floor plans into 3D models gives Zillow an edge over other real estate sites. The compa­ny says just the addition of 3D models is keeping people on the site longer.

Floored has a semi-automatic ap­proach for transforming 2D drawing to 3D models quickly, and they use traditional modeling and rendering to build 3D walkthroughs. They have built Luma, their own 3D engine based on WebGL, which allows 3D rendering and walkthroughs, but can also detect browsers that are not up to WebGL snuff and can gracefully degrade to a view the browser can handle. For the giant Hudson Yards project, Floored is building full-featured 3D walkthroughs. You can also add Floored to the grow­ing list of companies fascinated by the potential of VR.

Imagine you in 3D

It’s becoming easier to capture peo­ple and objects in 3D. Just a couple of years ago it took eleventy-seven pictures taken all around an object and stitched together. Or it took a high-priced struc­tured light scanner and some expertise. Artec, based in the Netherlands, has several products for 3D capture, and the company is specializing in creating 3D models of people. Their Scanify scan­ner can capture a person in 12 minutes and then process the scan in a half hour or so (see photo, page 1). The 3D prints from the scans look really good, and that’s because the software knows it’s dealing with a person, so when it en­counters anomalies in the scan it has a pretty good idea of how to fix it quickly and consistently. Artec also builds hand­held scanners including the Artec Eva, which is used for smaller models, and the Artec Spider, which is used in CAD applications. The company’s new Scanify booth is being promoted as a novelty item for retail spaces. The company is looking for development partners, and it sees two interesting markets ahead: digital characters for gaming or even communication, and it is looking next at digital manikins for online retailers.

By now, Intel’s RealSense is old news. Last September, Intel released an updated SDK to the public for use on Windows machines. The new SDK supports augmented reality, facial analysis, gestures, hand and finger tracking, and speech recognition. At Real, managing director for new business perceptual computing Mark Yahiro encouraged developers to have a look at Intel’s SDK. He showed off a smaller camera that can be used in tab­lets, and he showed updates to the SDK. The show-stopper was using a tablet to quickly scan a person and make a 3D model.

Dotproduct was founded by ex- Google employees who wanted to ex­plore what happens when 3D cameras are ubiquitous. It’s beyond cool that you can capture 3D content and create mod­els; it’s also practical. This data is deliv­ered with context so it can be measured. Tom Greaves of Dotproduct said that they’re finding applications for their mesh creation product for crime scene reconstruction. It’s a way to capture the scene quickly before anything gets dis­turbed. Another use is fast and accurate captures of construction sites to identify problems. Endangered heritage sites can be captured so people can experience them without damaging them or getting damaged, in the case of sites in conflict zones.

As a case in point, speaker Sarah Kenderdine at the University of New South Wales described projects to cap­ture environments such as the Dun­huang habitats in the Gobi Desert. The caves are endangered by tourist traffic, and a digital recreation has been built to let people see what the caves are like without going inside them. It’s easy to suspect that years after people trek to the desert and see the digital installa­tion, they’ll be convinced they were ac­tually in the caves as the memory does its magic melting-watch effect.

Thinkbox is a company that has come out of the entertainment world to offer a product to AEC professionals. Their Sequoia product is able to quickly and automatically mesh point clouds to create models that can then be used in 3D engines like Unity or for as-built records of buildings under construction. Again, the beauty of this technology compared to photographs or video is that the data is measurable and can be used in design programs, games, for VR walkthroughs, etc. The Sequoia product has not been officially released.

The emergence of hybrid approaches

The structured light cameras like those from Apple PrimeSense, Intel RealSense, Artec, and Mantis, to name a few, are getting all the press because they’re going to be the camera that we all have. However, laser approaches can be more powerful for large-scale cap­ture and for capture under difficult cir­cumstances. Topcon, for instance, is one of the old-school companies in the lidar business. Topcon’s Eduardo Falcon said that the company is looking at ways to combine data. Topcon built the scanners used for the first Google Street View cameras, but in the future he sees their customers getting an aerial view using UAVs in the sky and using lidar to drive around the outside. Inside, surveyors may well use structured light scanners to capture rooms.

Stuart Woods from Leica, part of Hexagon’s large metrology empire, adds time as the next variable. The whole world is being captured, but the data is not being refreshed fast enough. He says that we can have actionable data when we have capture–compute–create–repeat. There are potential data gather­ing approaches like cars driving on the highway and picking up data, sensors in the house, and regular fly-overs by UAVs to enable a better job of moni­toring the condition of highways, or se­niors at home, and of crops in the field. And inevitably, even as you read that, the questions popped up: Wait, who owns the information those sensors are picking up. Is it for sale? What about the scan of me I made at the depart­ment store—can the insurance company use it to tell me I’m gaining weight?

Recap and Memento

In the end, perhaps the Real 2015 show was one of the most elaborate launch events ever staged—though probably not the most expensive, since Autodesk was able to line up a respect­able number of exhibitors. Autodesk brought in the worldwide CAD press to show off Memento and Recap. Both products are designed to enable easier 3D capture and modeling, but Recap Pro is the company’s professional tool. It is sold as a monthly subscription, and it is designed for complex capture proj­ects that require registration to bring the data together.

Autodesk’s Memento has been a lab product, and now the product is making its next step toward product-hood; it’s being offered as a free Beta release for reality capture and processing. It is de­signed to be used with point clouds and can handle huge amounts of data. The company says they’ve fed it over 400 million polys and it has not choked. So far, Memento is being offered as a tool for people to experiment with build­ing models from point clouds, though Autodesk says it can also be used as a companion product for its Recap Pro. Memento can take in point cloud data created with photos, image-based mod­elers, from structured light scanners (but at the moment it only supports Artec’s Eva and Spider), and data from laser scans from processed by Recap Pro. Me­mento creates a mesh, which can then receive a texture for a realistic model. Models can be delivered as solid with wireframe, textured, and textured with wireframe mode. It is a program that sits on a computer, Windows for now, but the data is sent to the cloud for pro­cessing.

“What’s new,” says Autodesk VP Brian Matthews, who is a group CTO for the Information and Modeling Plat­form Group (IPG), “is how easy it is.” Autodesk has added as much automa­tion as it can to improve ease of use and performance, but, Matthews would argue, it has also added as much intel­ligence as possible to ensure models are usable. Memento includes cleanup tools and diagnostics to enable com­mon large-mesh issues such as particles, holes, spikes, and tunnels. Thus, the model can then be sent to a 3D printer or used as a digital asset. Memento is an advance over Autodesk’s 123D Catch, which was released for the iOS platform a few years ago. That product was lim­ited to how many pictures you could use to create a model. Memento access a variety of inputs and huge amounts of data.

Tatiana Dzambazova, an evangelist at Autodesk for reality capture and fabrication, demonstrated the program, telling the audience, it’s too easy for geeks. She noted that when average people walk around an object taking pictures, they don’t realize they are ac­cidentally gathering data from stuff they don’t want: the background, what’s be­hind the picture. So one of the simplest and first things Memento lets you do is select the object you really wanted to scan and cut away all the 3D trash you accidentally captured. Then more sophisticated tools can come in to play including re-topologizing tools, decima­tion, and the ability to isolate areas that may need extra cleanup.

Ironically for Dzambazova, who pre­fers the Mac platform, Memento is a Windows product. She promised the audience there will be a Mac version be­cause she is a great believer in the Mac platform, and certainly for hobbyists the iPad and iPhone have provided the first capture medium for 3D point clouds.

Matthews said that Autodesk is also going to be working on ways to com­bine data from laser scanners as well as from photogrammetry methods and structured light. Laser scanners can op­erate in dark places and itcan be better for indoors scanning. Structured light might also work better in challenging light conditions. So sometimes, the best solution will be a combination of data.

What was interesting at the Real conference was that so much of the leading-edge technology was practically free. Blame that on the PrimeSense SDK, which has so inspired so many tinker­ers. There were quite a few companies offering these tools. The difference is in the size of the objects or the sites being captured. Recap Pro is for large projects for professional engineering firms. It cur­rently sells for $450 a month. The busi­ness model is for companies that may need the scanning software for a product but would not need it all the time. A new version of Recap is on the way, and it’s possible pricing models or perhaps the packaging might change. Likewise, Me­mento might wind up being offered as more of a service rather than a product. It’s a competitive sea right now, and Au­todesk is just one more swimmer in it.