Monte Carlo (MC) linear solvers are fundamentally based on the ability to estimate a matrix-vector product, using a random sampling process. They use the fact that deterministic stationary iterative processes to solve linear systems can be written as sums of a series of matrix-vector products. Replacing the deterministic matrix-vector products with MC estimates yields a MC linear solver. While MC linear solvers have a long history, they did not gain widespread acceptance in the numerical linear algebra community, for the following reasons: (i) their slow convergence, and (ii) the limited class of problems for which they converged. Slow convergence is caused by both, the MC process for estimating the matrix-vector product, and the stationary process underlying the MC technique, while the latter is caused primarily by the stationary iterative process. The MC linear algebra community made significant advances in reducing the errors from slow convergence through better techniques for estimating the matrix-vector product, and also through a variety of variance reduction techniques. However, use of MC linear algebra is still limited, since the techniques use only stationary iterative processes resulting from a diagonal splitting (for example, Jacobi), which have poor convergence properties. The reason for using such splittings is because it is believed that efficient MC implementations of more sophisticated splittings is not feasible. Consequently, little effort has been placed by the MC community on addressing this important issue. In this thesis, we address the issue of improving the iterative process underlying the MC linear solvers. In particular, we demonstrate that the reasons for considering only diagonal splitting is not valid, and show a specific non-diagonal splitting for which an efficient MC implementation is feasible, even though it superficially suffers from the drawbacks for which non-diagonal splittings were not considered by the MC linear algebra community. We also show that conventional techniques to improve deterministic iterative processes, such as the Chebyshev method, show promise in improving MC techniques too. Despite such improvements, we do not expect MC techniques to be competitive with modern deterministic techniques to accurately solve linear systems. However, MC techniques have the advantage that they can obtain approximate solutions fast. For example, an estimate of the solution can be obtained in constant time, independent of the size of the matrix, if we permit a small amount of preprocessing. There are other advantages too, such as the ability to estimate specific components of a solution, and latency and fault tolerance in parallel and distributed environments. There are a variety of applications where fast, approximate, solutions are useful, such as preconditioning, graph partitioning, and information retrieval. Thus MC linear algebra techniques are of relevance to important classes of applications. We demonstrate this by showing its benefits in an application to dynamic load balancing of parallel computations.
