The carbon footprint of astronomical research is an increasingly topical issue with first estimates of research institute and national community footprints having recently been published. As these assessments generally do not take into account the contribution of astronomical research infrastructures, we propose to complement them by providing an estimate of the contribution of astronomical space missions and ground-based observatories using greenhouse gas emission factors that relate cost and payload mass to carbon footprint. We find that use of astronomical research infrastructures dominates the carbon footprint of an average astronomer. Comparison of our findings with the socio-economic pathways that, according to the Intergovernmental Panel on Climate Change (IPCC), are compliant with keeping the global average temperature rise below levels of 1.5°C or 2°C suggests that drastic changes are needed on how astronomical research is conducted in the future. Specifically, continuous deployment of ever more and larger astronomical research infrastructures is clearly not sustainable. We argue that a new narrative for doing astronomical research is needed if we want to keep our planet habitable.