Scanning in AR
Overall
We used AR Kit2 to build an IOS AR application. The application can detect real-world physical points and recognize planar surfaces (horizontal and vertical surfaces). The surfaces are positions to place whatever virtual objects or effects we want. We put a portal environment, a door that users can enter into the exhibition room of Caribou in the Natural History Museum. The exhibition model was created by the photogrammetry method. Users can go into the portal and see the caribou closely and feel the snow environment. The realistic visualization can make users feel these caribous are still alive and exist physical stay around them. We think the feeling of using this AR application gives users to rethink the climate change
Plan detection and Place Object
We used AR Kit to build an IOS AR application. It enables users to detect real-world feature including point clouds and surfaces. After the surfaces are showing, users can put the preset virtual object. The example below shows a tiger put on the detected table.
The placed object was finally changed to the portal prefab, which can put the entire caribou exhibition room on a detected open space.
​
AR portal prototype
An AR portal is an entrance to the virtual world for users to walk in. It usually the shape of a door or window. There is a transparent shader invisible but will trigger the material visible when users walking enter the door, which will give them a completely different experience.
We used stencil combined with customized Portal Shader Script to realize the portal effect. Basically, we applied our Portal shader to the objects’ materials, to let it invisible (equal with stencil ref) but exposed from portal when the camera was out of the portal. And the portal door was a trigger, so when the camera into the portal, all the materials in the portal would change to visible (NotEqual with stencil ref).
Materials are only visible through the Portal door (stencil) where there is a portal shader
Setting materials to NotEqual will be always visible
Model Optimization
Inside the portal, we put the caribou model as an asset in Unity. The caribou model was generated by the photogrammetry method using the photos taken in the Exhibition Room of the NHM and is quite useful because the OBJ file is only 15 MB with texture. It means the portal asset uses tinny memory to run the AR application. after the test showed below turned out our OBJ file did not ruin the AR portal application. The system is very fast. which uses tinny memory to run the AR application.
We created a new shader for the Caribou to realize the portal door shader to trigger the caribou material appearing or not. Also, we rescale the caribou size to the appropriate size as it should be. The example below shows the correct size of the caribou in the portal when using the built application.
Using photogrammetry model in the portal
Addition of weather effect
We added the snow effect to our final demo application. Using different materials in the particle system and adding lighting and depth made the snow effect look more realistic. For example, snowflakes fall can stay on the ground for a while.
Weather effect