Adding someone to an out of office approver list, but it is grayed out and will not let me edit.

If the approvers set their own OOO through their employee user preferences, we do not allow to edit from the admin part of the application.

If you need to make an adjustment to an approvers OOO period/alternate approvers that the approver set up themselves via preferences - You as admin can delete their OOO period from Admin, then re-create a new period for them with updated period/alternate approvers.