Computer vision accelerates self-driving reaction workups from being automated to autonomous

Researchers in Canada and the US have created a computer vision system for workup procedures. The platform monitors multiple visual changes during processes such as solvent exchange distillation, crystallisation, solid–liquid mixing and liquid–liquid extraction to advance the real-time response capacity of automated systems.

Chemical research devotes significant time to monitoring and adjusting conditions of known and new reaction processes to maintain consistent yields, purities and compositions of reaction products. Self-driving laboratories, which combine artificial intelligence with robotic platforms, are helping to speed up such processes as well as improve their reproducibility and accuracy. Integrating self-driving labs with computer vision, a technique that captures, processes and analyses digital images of chemical reactions in real-time, can enhance them further still.

Now, Jason Hein and his research team from the University of British Columbia in collaboration with Pfizer have developed a platform that integrates computer vision, machine learning, real-time monitoring techniques and semi-automated laboratory reactors to automate and optimise workup processes. Called HeinSight2.0, the platform is an upgrade on the team’s previous system, HeinSight, and monitors physical outputs such as solid residues, liquid levels, homogeneity, turbidity and colour.

The original HeinSight system used simple logic to derive information from images to monitor and control liquid-levels across several chemical applications. ‘Heinsight2.0 is more contextual, recognising objects and surfaces, producing a more reactive system,’ explains Hein. It understands visual information, for example if a red to green colour change signals the end of a reaction, and cross-validates such data to respond in real-time.

‘People should be able to download the program script to similar hardware and it will work in the same manner ours did,’ Hein tells Chemistry World. ‘That’s why we released the entire code, everything is open source, anybody can pick it up and run it now and retrain the model to their experiments, super easy.’ In doing so, Heinsight 2.0 will increase the availability of reliable chemical workup data; a quest that physical organic chemist Niek Buurma, of Cardiff University in the UK, applauds. ‘In chemistry I think we need more reliable data,’ says Buurma. ‘Anything that automates collecting data is going to provide a good database of detailed information on workup processes that future analyses may go through and do machine learning and optimisation of.’

The team intends to design future HeinSight systems to connect past events with future predictions. ‘These change variables matter. We want the system to be able to react differently depending on what the history is,’ he says.

Enhancing automation not only ‘improves safety and reproducibility’, adds Buurma, but also improves equality and diversity in research. ‘If we have an individual who struggles with their vision, computer vision can assist. If we have individuals struggling with dyspraxia, automatically controlled pumps can assist them.’