Fix gem checks judging unused system gem homes (#1098, #1109)

Fix gem checks judging unused system gem homes (#1098, #1109) #1634