We need to make sure that destructors can't access the partially destroyed property. Do the same we do in HTs. Fixes oss-fuzz #36205.