Most modifiers in general need to go. This is a bad way to impose challenge. Anything that restricts or is a hinderance to what players can normally do is just a poor way to impose challenge. It needs to come from the enemie’s abilities and behavior and/or the mechanics if the activity has any.