Computer-guided retrosynthesis

When IBM’s Deep Blue supercomputer beat world chess champion Garry Kasparov in 1997, few chemists must have realised that this might signify a win for them too. But Bartosz Grzybowski did. Then a PhD student at Harvard University in the US, outside chemistry he plays the card game bridge avidly. Grzybowski’s opponents were interested in algorithms like Deep Blue’s. ‘I started thinking, why couldn’t this be done for molecules?’ he recalls. In particular, might similar algorithms help plan the strategy for making target molecules, which chemists call retrosynthesis.

Computer-aided retrosynthesis had been tried before, most notably by Nobel laureate organic chemist E J Corey. Yet those attempts were limited by the scale of the challenge. ‘I even went to one very famous chemist, and he told me “It cannot be done”,’ Grzybowski says. But he was undeterred.

The idea stuck with Grzybowski after he started his own research group, and they began describing chemical syntheses as networks in 2005. At first, they looked at previous chemical reactions, connected by a series of statistical laws describing how chemists approach making organic molecules. By 2012, they introduced scoring functions to evaluate and optimise existing syntheses, referring to starting materials that were then commercially available from fine chemical supplier Sigma–Aldrich, calling their tool Chematica. But Grzybowski wanted algorithms to improve synthetic routes using steps that were unprecedented in the chemical literature, like Deep Blue sought the best chess moves in entirely new games.

Yet published details of chemical experiments can have problems. There are many errors, and they are biased towards simple and successful experiments. So around 2010, Grzybowski and colleagues made the key decision to manually code rules describing reaction mechanistic classes, including functional groups with which they are incompatible and that must be protected. Grzybowski’s team from the Polish Academy of Sciences published the first pathways found from scratch by Chematica in 2016. But it was only in 2020 that they claimed victory over their famous chemist doubter, when Chematica produced routes to complex natural products.

Organic chemistry is not just about memorising rules, it’s also about learning its subtleties

As this story was unfolding, in 2017, German chemistry giant Merck KGaA, which owns the Sigma–Aldrich business, bought Chematica and renamed it Synthia. It decided to do this after testing automated retrosynthesis on seven compounds Sigma–Aldrich wanted to make. ‘Over the course of three days, we had a crash course in how to use it,’ explains Lindsey Rickershauser, who is now sales and marketing manager for Synthia. ‘We had rules surrounding it, so that we would mimic the pressure scientists and chemists are under.’ In each case Synthia found new synthetic routes to entirely new products, increased experimentally obtained yields, lowered costs and/or reduced the number of synthetic steps needed.

Now part of Merck, Synthia incorporates over 100,000 hand-coded rules, each of which required a long search, potentially taking a few weeks. The potential value to the Merck conglomerate alone would have made the investment worthwhile, says Rickershauser. Yet today it is available for chemists elsewhere to licence, and it i addsts s used by ten of the top 20 pharmaceutical companies, Rickershauser adds. ‘Most of the chemists that use Synthia, they’re not necessarily pulling out an entire pathway from the software to execute right from beginning to end,’ comments Rickershauser. ‘They’re finding inspiration and disconnections that they never would have thought of.’

Yet Synthia now faces a number of challengers.

Limitation check

In 2014, Marwin Segler approached automated retrosynthesis from a slightly different direction during his PhD at the University of Münster, Germany. He sought to avoid the need for experts to teach retrosynthesis algorithms organic chemistry rules. Instead, Segler turned to machine learning. Traditionally, purely rule-based pattern-matching techniques proposed to automatically teach computers chemistry were unable to work robustly enough for retrosynthesis. But Segler adopted new techniques better suited to chemical challenges. ‘We found that applying machine learning for retrosynthesis worked surprisingly well and resolved some of the long-standing challenges,’ Segler says. ‘Intuitively, this makes sense. Organic chemistry is not just about memorising rules, it’s also about learning its subtleties from experimental data.’

That work has since proven influential, with other groups adopting similar approaches. Scientists like Segler, now at Microsoft Research in Cambridge, UK, are seeking to develop systems that can do what Synthia can without being taught by experts. They go beyond retrosynthesis, to challenges that might trouble human chemists. From looking backwards to figure out how to make target molecules, they are already looking forwards and predicting reaction outcomes.

The retrosyntheses produced by the algorithm were indistinguishable to those made by humans

Segler also compares retrosynthesis to chess. The longest tournament chess game was 269 moves, but there are a relatively small number of options for each move. Retrosynthesis – starting with the end product you want to make and working backwards to the starting materials through multiple reactions – differs on both points. 20 moves, or intermediate steps, would be too long for most retrosyntheses, but at each step there may be thousands of potential synthetic routes. Machine learning involves teaching a neural network by feeding it data. Segler’s algorithm first learned the rules around chemical reactions from millions of reactions in the Reaxys database. Then, to do retrosynthesis, it searches through the steps it has learned. As theoretical chemists, the Münster team faced a practical problem in testing how well this worked. They couldn’t easily just try out the proposed routes, so what could they do?

The answer would come from one of the pioneers of computing: Alan Turing. Turing famously proposed a test to tell whether a computer is able to think like a human being by holding a conversation with a real human. In this case, the Münster scientists tested whether human chemists could tell whether the retrosynthesis was produced by a computer. ‘If you look at review papers from the 2000s, the criticism you always had was that you get a lot of unreasonable routes out of such systems, which for expert organic chemists are very easy to spot,’ Segler says. The retrosyntheses of some typical medicinal chemistry targets produced by the Münster team’s algorithm were indistinguishable to those made by humans. ‘That was surprising,’ Segler says.

When it was published in 2018, the Münster team’s research ‘spurred a renewed interest in automated retrosynthesis’, according to Esben Bjerrum, who works for pharma giant AstraZeneca in Gothenburg, Sweden. He joined the company shortly afterwards, and because the original system wasn’t freely available, worked with his colleague Samuel Genheden and PhD student Amol Thakkar to develop one that was. They call their version AiZynthFinder, and Bjerrum highlights the advantage of openly available source code.

‘We have also seen the integration into RetroBioCat, a tool aimed at bioconversion prediction, but where the option to use standard reactions from AiZynthFinder was added as an alternative,’ Bjerrum says. ‘The tools we are developing are already usable. However, we continue to work on improving them.’ A Japanese Twitter bot also uses AiZynthFinder to retweet a predicted synthetic route if you send it a structure in a Smile string format.

There is an exponentially growing rate of publications in chemistry. It is difficult to imagine keeping up in the long term.

AiZynthFinder has also been integrated into a version of Automated System for Knowledge-based Continuous Organic Synthesis (ASKCOS), a tool developed at the Massachusetts Institute of Technology, US. Emerging from the 2016 Darpa-funded Make-It programme, ASKCOS is as much about methods of automated retrosynthesis as it is about results, says MIT’s Connor Coley. ‘The way we approach it is quite different from the more expert-driven systems,’ he says. ‘The goals are the same in enabling access to large chemical spaces and making it faster, cheaper, easier to synthesise new structures or to propose better ways to synthesise old structures. But then it’s about learning the limitations of the techniques, advancing the algorithms and computational methods we use to process, characterise and learn from experimental data on organic reactions.’

Scale models

ASKCOS is part of the Machine Learning for Pharmaceutical Discovery and Synthesis Consortium, led by MIT and involving 15 drug companies. In 2019, the consortium published a paper integrating ASKCOS into automated robotic synthesis, revealing both its capabilities and limitations. A fundamental challenge for machine learning is generalising from noisy data with mistakes and big gaps in it, Coley adds. ‘Of course, humans can do extrapolation, and write down what we believe to be the rules and the scope of a reaction,’ he says. ‘We are working on ways to try to infer that just from data.’ That’s important because Coley sees scenarios where automated inference will be faster and more scalable than the human-powered approach. ‘There is an exponentially growing rate of publications in chemistry,’ he says. ‘It is difficult to imagine keeping up in the long term.’

For seldomly used reactions there’s often not enough data to build robust models 

Some researchers use machine learning techniques to predict which reactions will work, including their conditions, and potentially devising entirely new reaction classes. Again, they’re moving from replaying past chess moves to inventing entirely new ones. For this application ‘there are likely some aspects where expert systems are going to be even more tedious to deploy than in retrosynthesis’, Coley says.

Bjerrum cautiously echoes this outlook. ‘The balance will likely tip from expert rule-based systems towards methods that scale better with the ever-increasing amount of data,’ he says. He notes, however, that data often needs ‘a lot of curation and cleaning in order to be useable for deep learning’. ‘For seldomly used reactions there’s often not enough data to build robust models and there the rule-based system have an edge, as it’s possible to build in the knowledge of the chemical context,’ Bjerrum says. ‘But why not combine? I would love to try out deep learning with the hand-curated templates that are used in Chematica.’

From his perspective at AstraZeneca, Bjerrum doesn’t have the impression that chemists use automated retrosynthesis tools commonly, preferring their own experience and knowledge. ‘More awareness of the benefits and ease-of-use of automated retrosynthesis tools is needed,’ he says. ‘The tools have a knowledge base of millions of reactions and a full overview of the available stock that can be searched automatically. Chemists that use our tool thus get a good overview of potential reactions and building blocks to use in the context of a given target molecule, even if the predicted routes may not contain the exact one that’s finally used.’

Yet this year significant steps taking self-taught systems further have been made by none other than Deep Blue’s developer, IBM. Since 2017, the company has applied machine learning to chemistry using technology similar to automated translation. Training algorithms on chemistry patent data, IBM could automatically extract reaction rules and then predict reaction outcomes from their reactants. It made the resulting RXN for Chemistry tool available online in 2018. In 2019 the company coupled these models with other algorithms to do retrosynthesis. RXN uses the reaction prediction algorithm to help assess the retrosynthesis results, explains IBM’s Alessandra Toniato, checking that the proposed route will produce the target molecule.

This can automatically clean up dirty datasets, as models struggle to learn reactions that contain incorrect elements. IBM instructs models to watch for hard-to-learn reactions and remove them from the original dataset. ‘One interesting experiment that we did was to take a cleaner data set and introduce some noise,’ Toniato says. ‘Placing random molecules where they shouldn’t be and replacing the correct products with ones that were similar. And the model, by applying this unassisted technique, was really able to detect this kind of noise.’

Performance pressure

In 2020, IBM extended the approach to synthesise molecules on robots automatically and remotely. It created a data set of operations like liquid–liquid extractions and filtering from 700,000 reaction records. Researchers then used those records to train a new machine learning model to ‘take a chemical equation and convert that to a series of steps that can in principle, be executed directly on the RoboRXN robot,’ explains IBM’s Alain Vaucher. The system is currently free to use, adds team leader Teodoro Laino. ‘Everybody has access to the simulator of the robot,’ he says. ‘If you wish to access the real hardware, then you need a key from us. If you like it, there is the possibility to have a similar installation on your premises.’

To date there are 26,000 users for RXN overall, says Vaucher, who together have made 3.7 million reaction predictions. IBM has used it to develop new carbon capture and semiconductor manufacturing materials. In the UK, the Diamond Light Source synchrotron will begin using RXN’s models with its own robots starting in the first half of 2021.

There is no single magic bullet

Laino concedes that retrosynthesis tools based exclusively on machine learning lag behind Synthia on performance. ‘Synthia has been built on a large amount of literature and data sources,’ he says. ‘The performance that organic chemists experience, it’s really the outcome of the knowledge that is in the dataset.’ Yet now IBM is set to partner with a European publisher to train models with high-quality chemical data. ‘That is going to where we are going to have a comparison that is a little bit fairer,’ Laino says. High quality data and unsupervised rule extraction will allow IBM ‘to replicate Synthia in just a couple of days’, Laino adds.

Grzybowski is sceptical about the prospects of systems that exclusively use machine learning, calling the idea ‘unlikely, given how noisy the literature is and how much it is dominated by simple reaction types’. However, several teams, first Segler’s and Coley’s, and later also Grzybowski’s, have shown synergies between machine learning and expert rule-based systems. ‘There is no single magic bullet,’ Grzybowski says. ‘Yes, there is a place for AI in this, but there’s also a place for mechanistic knowledge, for quantum mechanics, for molecular mechanics.’ Ultimately, the matter of which technology is best at retrosynthesis is secondary, he concludes. ‘I don’t want to be attached to a buzzword, I want to solve the problem.’

Andy Extance is a science writer based in Exeter, UK