Augmented Reality Core Benchmarking on Chromebooks

Augmented reality is an enhanced version of reality where computer-generated images are superimposed on live direct or indirect views of the user's physical real-world environments, thus augmenting one’s current perception of reality. As both virtual and real worlds harmoniously coexist, users of augmented reality experience a new and improved natural world where virtual information is used as a tool to aid in everyday activities. With each step forward in the digital revolution, hardware engineers and software developers continue to refine their augmented reality technologies, making the line between real and virtual life. Refer to Figure 1 below to compare augmented reality to virtual reality [3].

Figure 1 Augmented Reality vs Virtual Reality

ARCore is Google’s Platform for building augmented reality experiences. Using different APIs, ARCore enables an Android-based smartphone to sense its environment, understand the world, and interact with information. It is supported in Android 7.0 (Nougat) or later desserts.

ARCore uses three key capabilities to integrate virtual content with the real world as seen through your phone's camera [2]:

1. Motion tracking allows the phone to understand and track its position relative to the world.
2. Environmental understanding allows the phone to detect the size and location of all types of surfaces: horizontal, vertical, and angled surfaces such as the ground, a coffee table, or walls.
3. Light estimation allows the phone to approximate the environment's current lighting conditions.

If we refer to Figure 2, ARCore works on the camera and sensors to integrate virtual content to the real world. The output is then viewed in the Chromebook display.

There are no standard benchmarking tools available for benchmarking ARCore. The camera and sensors have already been benchmarked separately while characterizing the SOC. The ARCore assumes the data provided by the camera and sensors are accurate. Now we need to measure the frames per second of the output generated by ARCore.

Our approach

Based on the various categories of AR, most of them are used in our day-to-day activities. This involves the projection of an object, viewing a place etc., where picture’s role becomes vital. The key metric in such cases is frames per second (FPS) calculation, also referred to as frame rate. Frame rate greatly impacts the style and viewing experience of a video. Different frame rates yield different viewing experiences. Choosing a frame rate often means choosing between such things as how realistic you want your video to look or whether you plan to use techniques such as slow motion or motion blur effects. For example, most feature films are shot and viewed at 24 fps; sports and video games contain busy scenes with a lot of motion.

Our first approach was to identify the apps that use ARCore functionality. The apps listed below have been selected for their popularity and high ranking. Refer to the table below for popular ARCore apps.

Table 1: Popular ARCore Apps

Google Expeditions

Google Expeditions is an immersive learning and teaching tool that lets you go on or explore AR objects. Explore historical landmarks, go down to the atomic level, get up close with sharks, even visit outer space!
In the classroom or with groups, Google Expeditions allows a teacher acting as a 'guide' to lead classroom-sized groups of 'explorers' through VR tours or show them AR objects. Guides can use a set of tools to point out interesting things along the way.

Google Translate

Google Translate allows students to type out a different language to be translated to English. Two new features are enabled by combining ARCore with Google Translate. The first feature allows students to use the camera function and take a picture of the text or a sign and have it translated in real time. The other feature is being able to translate spoken words in real time.

The two benchmarking tools for FPS measurement for ARCore output that we used are:

1. cafac2 tool
2. Gamebench (https://www.gamebench.net/)

These are user-friendly and easy-to-use tools. The procedures for setting up the tool and capturing the FPS is documented in detail by the tool vendors.

The standard steps for the cafac2 tool are shown below:

1. Push the FPS app to the Chromebook using the script provided in the test bundle.
2. Execute the script and launch the application.
3. Run the AR app on your target Chromebook. Ensure that content picked up has overlapped images and runs for at least 2 minutes.
4. The FPS rate is displayed in parallel on the screen when the app is executing.

For the Gamebench tool:

1. Push the Gamebench app and other jar files to the Chromebook.
2. Execute the jar fie, which launches the Gamebench app automatically.
3. Now launch the AR app in your Chromebook.
4. The AR app is listed In the Gamebench dashboard.
5. Run the AR app on your target Chromebook. Ensure that content picked up is having overlapped images and runs for at least 2 minutes.
6. The FPS rate is displayed in parallel on the screen when the app is executing.

Results

Our evaluation platform is the Google Pixel Slate, which uses an Intel® m3-8100Y processor with 2 cores. The processor base frequency is 1.05 GHz and can reach up to 3.4 GHz in Turbo mode. The memory available in the device is 8 GB. The latest ChromeOS version, R74 with Android Pie, is loaded on the device. We have ensured that “Internet Speed Test” is executed before collecting the data to confirm that the internet bandwidth is the same while executing the tests. The apps are side loaded to the system and tests are applied.

Refer to Figure 3 for FPS display.

The table below provides the FPS data for different use cases for Google ARCore apps.

Table 2: FPS Data for Apps

Conclusion

The goal was to provide an overview of the ARCore app and a performance measurement of ARCore in Chromebooks. In summary, we can conclude that ARCore performance on the Chromebook is reflected by good FPS rates. Email to ask questions, or discuss issues: jaishankar.rajendran@intel.com biboshan.banerjee@intel.com

Test configuration

ChromeBook: Google Pixel Slate

Software: Android 9.0, OpenGL ES GLSL ES 3.1 Mesa, Vulkan 1.0.76 Support

Hardware: Intel® m3-8100 Processor, 2x3.4 GHz CPU, 8GB RAM

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com.

References

1. “AR are perfect for Chromebooks", [Online]. Available: https://www.edsurge.com/news/2018-05-15-these-new-web-based-ar-and-vr-tools-are-perfect-for-chromebooks
2. "AR Concepts," [Online]. Available: https://developers.google.com/ar/discover/concepts
3. "Augmented Reality," [Online]. Available: https://learningenglish.voanews.com/a/augmented-reality-versus-virtual-reality/3844772.html