As part of the International Muon Collider study, a beam vacuum window is being developed at CERN. It is required for the final cooling, where the charged particles travel from the vacuum chamber to the absorber; here, the beam loses momentum to cross a second window entering in a RF cavity that increases the longitudinal momentum. The best absorber for the final cooling is hydrogen. As the absorber should be installed inside a high field focusing solenoid, the hydrogen density should be as high as possible, ideally liquid or high pressure gas, to have a reasonable solenoid length. To evaluate the performance of the window, it is necessary to study the tightness at cryogenic temperatures, resistance to burst, high temperature and beam-induced damage. The main objective of the proposed work is to design and validate a versatile bulge test setup for the mechanical characterization of thin windows at different pressures and temperatures to cover all operating conditions, from 77 K to 293.15 K and ideally above. Due to the low thicknesses, a non-contact measuring technique based on a confocal chromatic sensor is proposed.