Keio University: NICT’s 7 Predictions for Monte-Carlo Optimization in 2026

Researchers at Keio University and the National Institute of Information and Communications Technology (NICT) present a Monte-Carlo Compressive Optimization algorithm, a new method to tackle combinatorial optimization problems, including Black-Box or complicated objective functions. This method estimates generalized moments and repurposes a greedy algorithm from compressive sensing to find the global optimum. The team provides numerical results demonstrating that their method is competitive when compared with dual annealing. An end-to-end open-source implementation is available, enhancing its practicality and allowing researchers to readily test and build upon their findings for a range of applications from engineering to finance.

Compute a sample of f called S_f

The team’s work centers around a method designed to efficiently sample potential solutions within a complex problem space, offering an alternative to existing methods. Specifically, the method computes a sample of f denoted as S_f, initiating the process by drawing a sample ℐ of n indices from a uniform distribution, a step crucial for reducing computational load. This sampling operator, S_ℐ, then acts on the input f, effectively selecting a subset of data points based on the generated indices. The method’s core innovation lies in repurposing a technique traditionally used for reconstructing signals from limited data to navigate the challenges of combinatorial problems where the number of possible solutions grows exponentially. The researchers define the sampling operator S_ℐ as mapping f(x) to f(x)δ_x,y∈ℐ, where δ_x,y∈ℐ represents the Kronecker delta, ensuring that only elements corresponding to indices within the sampled set ℐ are retained.

This selective approach allows the algorithm to focus on the most promising areas of the solution space, potentially accelerating the optimization process. Comparing the method to dual annealing provides a direct comparison point for evaluating the algorithm’s performance and efficiency. The team has made an end-to-end open-source implementation publicly available, which industry leaders predict will foster rapid experimentation and collaborative development. This accessibility lowers the barrier to entry for researchers and practitioners, allowing them to readily test and adapt the method to their specific problems. The availability of the code, coupled with the algorithm’s unique approach to sampling, positions it as a potentially valuable tool for tackling complex optimization challenges across diverse fields, from logistics and scheduling to machine learning and materials science.

Apply a hard thresholding operator T_t such that in T_tSf all values S_f(x) lower than a given t become zero

Industry leaders predict a growing emphasis on algorithmic pruning techniques, specifically the application of hard thresholding operators within optimization algorithms. The method involves applying a threshold, denoted as T_t, to the output of a sampling operator S acting on an input f; any resulting value S_f(x) falling below a predetermined threshold t is simply set to zero. This selective filtering is anticipated to become increasingly prevalent as computational demands escalate across diverse fields, from machine learning to logistics. Researchers at Keio University, including Baptiste Chevalier, Shimpei Yamaguchi, Wojciech Roga, and Masahiro Takeoka, have demonstrated the utility of this technique within a newly developed algorithm. The core principle, as described in their work, centers on reducing the computational burden by focusing on the most salient features of a problem. The team explains the fundamental mechanism for discarding less significant data points. An end-to-end open-source implementation is available to use their method, and they provide numerical results demonstrating that their method is competitive when compared with dual annealing.

Apply a sketching map (also known as a measurement map) Φ to get y = Φ(T_tSf)

This year will see increased attention to this technique as researchers explore its potential to accelerate solutions for complex problems. The core of their method involves applying a measurement map, Φ, to a transformed input f, creating a y that encapsulates key information about the problem’s underlying distribution. This process effectively distills the essential characteristics of the optimization challenge into a more manageable form, enabling faster and more efficient exploration of potential solutions. The team’s innovation lies in repurposing concepts from compressive sensing to address challenges traditionally tackled with different methods. Their approach differs from conventional techniques by focusing on estimating generalized moments of the probability distribution represented by f, captured within the sketch vector y. According to the researchers, the sketch vector contains an “empirical estimate of some generalized moments” which allows for a targeted search of the solution space.

This contrasts with dual annealing, against which the new algorithm has been benchmarked, offering a direct comparison point for performance and efficiency. An end-to-end open-source implementation is available to use this method, and is anticipated to foster rapid innovation and collaboration within the field, as others can readily integrate and adapt the sketching map technique into their own optimization pipelines. The resulting sketch vector, y, provides a condensed representation of the problem, allowing for focused computational effort and potentially unlocking solutions to previously intractable challenges.

Apply a decoding procedure Δ to y and get f̃ = Δ(y)

Industry leaders anticipate increased interest in decoding procedures borrowed from the field of compressive sensing to tackle increasingly complex combinatorial optimization problems. This year will see a growing trend of cross-disciplinary innovation, as algorithms proven effective in one domain are adapted to solve challenges in seemingly unrelated areas; the team’s approach leverages the efficiency of compressive sensing to navigate vast solution spaces. The core of this method, as described in their work, involves applying a decoding procedure, represented as Δ to y to obtain an estimate of f, expressed mathematically as f̃ = Δ(y). The researchers provide numerical results demonstrating that their method is competitive when compared with dual annealing. Looking ahead, experts anticipate that this type of selective optimization will become increasingly prevalent as computational demands continue to rise.

The algorithm’s reliance on concepts from compressive sensing, specifically, the repurposing of methods like Matching Pursuit and Orthogonal Matching Pursuit, suggests a shift towards more efficient sampling strategies. An end-to-end open-source implementation is available to encourage further exploration and refinement of this promising technique, potentially unlocking new capabilities in areas like logistics, scheduling, and resource allocation.

The largest value of f̃ is used as the maximum estimate

Industry leaders predict a significant shift in how combinatorial optimization problems are approached, moving beyond reliance on techniques like dual annealing and embracing methods inspired by signal processing. This algorithm doesn’t seek an exact solution immediately, but instead focuses on efficiently estimating the maximum value of a function, denoted as f̃, which is then utilized as the maximum estimate. The core innovation lies in approximating generalized moments, linking it to compressive sensing, statistical compressive learning, and Monte Carlo methods. The team explains that, given a sufficient number of samples, the average of a function g with respect to a probability distribution f can be approximated by sampling from a uniform distribution and evaluating the density function f at those points. This allows for a computationally efficient way to explore the solution space, particularly crucial for complex problems where exhaustive search is impractical.

The choice of a is paramount; functions satisfying the Restricted Isometry Property, commonly used in compressive sensing, are strong candidates, as are structured deterministic measurement functions. The researchers repurposed greedy methods from compressive sensing. They contend that emphasizing sparsity, identifying the most significant elements of the function, yields better results than focusing on strict constraints, a phenomenon known as overfitting in learning theory. The algorithm’s effectiveness is demonstrated on a class of objective functions where information is concentrated in a limited set of rules.

We can define the following vector: f_ℛ = ∑_r∈ℛω_rF_N, as a linear combination of the transform of all rules in ℛ and thus redefine the function f_ℛ(x) as the x-entry of the vector f_ℛ

This year will see increased exploration of cross-disciplinary algorithmic approaches, leveraging tools from seemingly unrelated fields to unlock new computational capabilities. The core of this advancement lies in a redefined mathematical framework. The team defines a vector, represented as f_ℛ and expressed through equation (4), as a linear combination of transformations applied to a set of rules denoted as ℛ. This allows them to redefine the function f_ℛ(x) as a specific entry within that vector, effectively transforming the optimization problem into a vector manipulation task. This seemingly abstract step is crucial, as it enables the application of compressive sensing principles. Physicists will recognize similarities between this diagonal form of the cost function and a generalized version of the Ising problem, where the value of each weighting factor depends on the spin configuration. The researchers demonstrate that the cost functions are often present in NP / NP-complete problems.

However, their Monte-Carlo method, benchmarked against dual annealing, shows promising results. “Indeed, for the quintuplet method, the distance to the optimum reduces in a manner similar, or faster, to the annealing method,” the team reports, suggesting a potential performance advantage, particularly with limited computational resources. The team has made an end-to-end open-source implementation available through the TrOMA library, which integrates the MCCO method, lowering the barrier to entry for wider adoption and further research.

A smaller variance and higher mean values both lead to increasing success probability, justifying the use of the thresholding step in Algorithm 1

Central to MCCO’s efficacy is the thresholding step within Algorithm 1, a design choice now rigorously justified by recent analysis. The team’s work reveals that a smaller variance coupled with higher mean values in the sampled data directly correlates with an increased probability of finding the optimal solution. This principle underpins the algorithm’s ability to efficiently navigate challenging optimization problems. Specifically, the analysis focuses on scenarios employing structured sketch matrices, such as Quadruplets or Quintuplets, where generalized moments become estimations of Walsh-Hadamard coefficients. “A smaller variance and higher mean values both lead to increasing success probability, justifying the use of the thresholding step in Algorithm 1,” explains the research. The researchers provide numerical results demonstrating that their method is competitive when compared with dual annealing. The practicality of the algorithm is enhanced by the ability to tune the heuristic parameters to the available computational resources.

An end-to-end open-source implementation is available to use the method. This accessibility will likely accelerate the application of MCCO to diverse fields, including reinforcement learning and the search for ground states in Ising Hamiltonians, potentially even bridging the gap between classical and quantum computing approaches to complex optimization.