Random search methods are implemented to solve the wide variety of the large-scale discrete optimization problems when the implementation of the exact solution approaches is impossible due to large computational demands. Initially designed for unconstrained optimization, the variant probabilities method allows us to find the approximate solution of pseudo-Boolean optimization problems with constraints. Although, in case of the large-scale problems, the computational demands are also very high and the precision of the result depends on the spent time. The rapid development of the parallel processor systems and clusters allows to reduce significantly the time spent to find the acceptable solution with speed-up close to ideal. In this paper, we consider an approach to the parallelizing of the algorithms realizing the variant probability method with adaptation and partial rollback procedure for constrained pseudo-Boolean optimization problems. Existing optimization algorithms are adapted for the systems with shared memory (OpenMP) and cluster systems (MPI library). The parallel efficiency is estimated for the large-scale non-linear pseudo-Boolean optimization problems.