Propensity scores have been central to causal inference and are often used as balancing weights. Using estimated propensity score as inverse weights, however, may exhibit undesirable finite-sample performance. Since propensity score is originally proposed by mimicking a randomized trial, and that an important property of a randomization is that confounders are balanced among treatment groups, we construct weights that directly balance the mean and other functionals of the covariate distribution. We show that the estimators have desirable theoretical and numerical properties. We extend the covariate balancing procedure to mediation analysis. Since mediators are post-treatment variables that are not balanced even by randomization, a tilted balancing method is needed.