In multivariate regression, researchers are interested in modeling a correlated multivariate response variable as a function of covariates. The response of interest can be multidimensional; the correlation between the elements of the multivariate response can be very complex. In many applications, the association between the elements of the multivariate response is typically treated as a nuisance parameter. The focus is on estimating efficiently the regression coefficients, in order to study the average change in the mean response as a function of predictors. However, in many cases, the estimation of the covariance and, where applicable, the temporal dynamics of the multidimensional response is the main interest, such as the case in finance, for example. Moreover, the correct specification of the covariance matrix is important for the efficient estimation of the regression coefficients. These complex models usually involve some parameters that are static and some dynamic. Until recently, the simultaneous estimation of dynamic and static parameters in the same model has been difficult. The introduction of particle MCMC algorithms has allowed for the possibility of considering such models. In this thesis, we propose a general framework for jointly estimating the covariance matrix of multivariate data as well as the regression coefficients. This is done under different settings, for different dimensions and measurement scales.