A theoretical foundation for a generalization of the elliptic difference Painlevé equation to higher dimensions is provided in the framework of birational Weyl group action on the space of point configurations in general position in a projective space. By introducing an elliptic parametrization of point configurations, a realization of the Weyl group is proposed as a group of Cremona transformations containing elliptic functions in the coefficients. For this elliptic Cremona system, a theory of $\tau $-functions is developed to translate it into a system of bilinear equations of Hirota-Miwa type for the $\tau $-functions on the lattice. Application of this approach is also discussed to the elliptic difference Painlevé equation.