How Microsoft’s ‘Algorithm of Thoughts’ Brings Human-Like Reasoning to AI

In the ever-evolving realm of artificial intelligence (AI), language models have transitioned from understanding language to becoming versatile problem solvers, primarily driven by the concept of in-context learning.

Microsoft’s Algorithm of Thoughts (AoT) takes this evolution further, enabling human-like reasoning, planning, and math problem-solving in an energy-efficient way.

By using algorithmic examples, AoT unlocks language models’ potential to explore numerous ideas with just a few queries. In this article, we explore the evolution of prompt-based in-context learning approaches and delve into how AoT is transforming AI for human-like reasoning.

If you’re already familiar with in-context learning, standard prompting, and chain-of-thought prompting, feel free to skip ahead to learn how AoT ties these approaches together.

In-context Learning

In-context learning is a transformative process that aims to elevate language models from mere language experts to adept problem solvers. To understand this concept better, envision these models as language learning students in a school setting. Initially, their education primarily involves immersing themselves in vast amounts of text to acquire knowledge about words and facts.

But then, in-context learning takes these learners to the next level by enabling them to acquire specialized skills. Think of it as sending these learners to specialized training programs like college or trade school. During this phase, they focus on developing specific abilities and becoming proficient in various tasks such as language translation (for instance, Meta’s Seamless M4T), code generation, or complex problem-solving.

In the past, to make language models specialized, we had to retrain them with new data, during a process known as fine-tuning. This became difficult as models got more extensive and more resource-intensive. To address these issues, prompt-based methods have emerged. Instead of re-teaching the whole model, we just give it clear instructions, like telling it to answer questions or write code.

This approach stands out for its exceptional control, transparency, and efficiency in terms of data and computational resources, making it a highly practical choice for a wide range of applications.

Evolution of Prompt-based Learning

This section briefly overviews the evolution of prompt-based learning from standard prompting to Chain-of-Thought (CoT) and Tree-of-Thought (ToT).

? Standard Prompting

In 2021, researchers conducted a groundbreaking experiment, prompting a single generatively pre-trained model, T0, to excel in 12 different NLP tasks.

These tasks involved structured instructions, such as the one used for entailment: “If {premise} is true, is it also true that {hypothesis}? ||| {entailed}.”

The results were astonishing, as T0 outperformed models trained solely for single tasks, even excelling in new ones. This experiment introduced the prompt-based approach, also known as input-output or standard prompting.

Standard prompting is a straightforward technique where you present a few task-related examples to the model before seeking a response. For example, you can prompt it to solve equations like “2x + 3 = 11” (Solution: “x = 4”). It’s effective for simple tasks such as solving simple math equations or translation. However, as standard prompting relies on isolated instructions, it struggles with broader context understanding and complex multi-step reasoning, rendering it inefficient for excelling in complex mathematical problems, common sense reasoning, and planning tasks.

The limitations of standard prompting have given birth to CoT prompting, which addresses these limitations.

? Chain-of-thought (CoT) Prompting

CoT is a prompting technique that empowers large language models (LLMs) to tackle problems by breaking them down into a series of intermediate steps, leading to a final answer. This approach enhances the model’s reasoning abilities by encouraging it to respond to complex, multi-step problems in a manner that resembles a logical chain of thought.

CoT prompting proves particularly valuable in helping LLMs overcome reasoning tasks involving logical thinking and multiple steps, such as arithmetic problems and questions related to common-sense reasoning.

For instance, consider using CoT prompting to solve a complex physics problem, such as calculating the distance a car travels during acceleration. CoT prompts guide the language model through logical steps, beginning with the car’s initial velocity, applying the distance formula, and simplifying calculations. This illustrates how CoT prompting dissects intricate problems step by step, assisting the model in achieving precise reasoning.

? Tree-of-Thought (ToT) Prompting

In specific scenarios, however, solving problems can involve multiple approaches. Conventional step-by-step methods like CoT may restrict the exploration of diverse solutions. Tree-of-Thought Prompting addresses this challenge by employing prompts structured as decision trees, enabling language models to contemplate multiple pathways.

This method empowers the models to tackle problems from various angles, broadening the range of possibilities and encouraging creative solutions.

Challenges of Prompt-based Learning

While prompt-based approaches have undoubtedly bolstered the mathematical and reasoning prowess of language models, they come with a notable drawback—an exponential increase in the demand for queries and computational resources.

Each query directed towards an online language model like GPT-4 incurs a financial cost and contributes to latency, a critical bottleneck for real-time applications. These accumulative delays have the potential to undermine solution efficiency. Furthermore, continuous interactions can strain systems, potentially resulting in bandwidth constraints and reduced model availability. It’s also crucial to consider the environmental impact; persistent querying amplifies the energy consumption of already power-intensive data centers, exacerbating their carbon footprint.

Algorithm of Thought Prompting

Microsoft has taken on the challenge of improving prompt-based methods regarding cost, energy efficiency, and response time. They’ve introduced the Algorithm of Thought (AoT), a groundbreaking approach that reduces the need for many prompts in complex tasks while maintaining performance.

AoT differs from previous prompting methods by instructing language models to generate task-specific pseudo-code, akin to clear Python-like instructions.

This shift emphasizes utilizing the model’s internal thought processes rather than relying on potentially unreliable inputs and outputs at each step. AoT also incorporates in-context examples inspired by search algorithms like Depth First Search and Breadth First Search, aiding the model in breaking down intricate problems into manageable steps and identifying promising paths to follow.

While AoT shares similarities with the Tree-of-Thought (ToT) approach, it distinguishes itself through its remarkable efficiency. ToT often requires a multitude of Language Model (LLM) queries, occasionally numbering in the hundreds for a single problem. In contrast, AoT tackles this challenge by orchestrating the entire thinking process within a single context.

AoT excels in tasks that resemble tree-search problems. In these scenarios, the problem-solving process entails breaking the main problem into smaller components, devising solutions for each part, and determining which paths to delve into more deeply.

Instead of using separate queries for each subset of the problem, AoT leverages the model’s iterative abilities to tackle them in a unified sweep. This approach seamlessly integrates insights from previous contexts and demonstrates its prowess in handling complex issues requiring a deep dive into the solution space.

The Bottom Line

Microsoft’s Algorithm of Thoughts (AoT) is transforming AI by enabling human-like reasoning, planning, and math problem-solving in an energy-efficient manner. AoT leverages algorithmic examples to empower language models to explore diverse ideas with just a few queries.

While building upon the evolution of prompt-based learning, AoT stands out for its efficiency and effectiveness in addressing complex tasks. It not only enhances AI capabilities but also mitigates the challenges posed by resource-intensive querying methods.

With AoT, language models can excel in multi-step reasoning and tackle intricate problems, opening new possibilities for AI-powered applications.