1)  Login to Outlook Web at https://mail.office365.com
 

2)  Click on the Settings wheel in the upper right:


3)  Choose GENERAL on the left side bar, then NOTIFICATIONS.
 

4)  Make desired changes in the reactions section.

       

4)  Click the X to save and close