An app that lets people turn their hand-drawn structures into 3D augmented reality molecules has caused excitement among chemists on Twitter.

Users say that MolAR is not only instructional but also fun to use. ‘The app provides quick visualisation of the 3D structure of a molecule in augmented reality (AR), making it effortless for users to understand molecular structures,’ says team leader Todd Martinez of Stanford University, US. ‘This offers a far richer learning experience. Rather than static 2D structures on a page, the molecules appear in 3D on their desk.’

‘My personal favourite is the option to view molecular vibrations in 3D,’ says PhD researcher and app user Dmytro Kravchuk from the University of Iowa in the US. But the tool also offers other highlights such as the ‘draw’ and ‘recognise’ functions, he adds. ‘While still in early stages of development, these features allow users to draw simple organic structures either on the screen or on paper, scan the drawings and visualise the drawn molecules in 3D space.’

Programmed by Martinez’s colleague Sukolsak Sakshuwong, the app was first released for iOS in March and has now been updated to include visualising hand-drawn structures and molecular vibrations. The idea initially came from Martinez’s co-worker Umberto Raucci after he had created an Amazon Alexa app that enables voice control for quantum chemistry calculations.

The hand-drawn structure recognition is currently limited to hydrocarbons, points out Martinez. ‘We’re in the process of extending this to include heteroatoms,’ he says. The feature is based on a machine-learning program called ChemPix, created by Martinez’ colleague Hayley Weir. It converts the drawings into the computer-readable Smiles (simplified molecular-input line-entry system) format. ‘The recognition software was trained on a large dataset of structures with wavy lines, uneven bond angles and a variety of background and paper types,’ explains Martinez. But since only plain hydrocarbon backbones were used for the training, writing CH3 or Me for methyl groups may still confuse the neural network.

An image of a phone screen that shows grapes lying on a market stall table. On top of the grapes there's a 3D chemical structure of resveratrol

Source: © Todd Martinez/Martinez Team

The app can recognise items such as grapes and respond by displaying structures associated with it

Kravchuk notes that although the software can easily recognise poorly drawn structures of non-cyclic hydrocarbons, recognition of cyclic structures is still challenging if the handwriting isn’t perfect. ’On the other hand, recognition of physical objects is well-done and works beyond recognising a bottle of water as H2O and coffee as a caffeine molecule.’

Jet-Sing Lee, a postdoctoral researcher at the University of Tokyo in Japan, tried MolAr and says he might continue using it for teaching purposes – but also during coffee breaks with colleagues. ‘It’s rare to see chemistry-based apps that are well developed and fun,’ he comments. ‘It’s very cool that quite a wide range of organic-based structures are available, from simple molecules to larger carbon allotropes and biomolecules. An additional feature that should be included is the visualisation of metal-containing structures, such as organometallics. That would broaden the appeal to a wider range of chemists.’

Martinez says that new features that consider stereochemistry and metal-organic compounds are being developed and that an Android version of the app is underway too. ‘We plan to incorporate dynamic motion, chemical reactions and computation of quantum mechanical properties. This will allow the visualisation of molecular orbitals and calculation of spectra.’

Topics