Where Azure MFA is only included in the paid Azure Active Directory Premium subscriptions (P1/P2 and EM+S suites), there is a free version for the Office 365 apps.
It is always a good idea to enable multi factor authentication, in case your credentials get stolen, the thief will not be able to use them because of the 2nd authentication factor. Microsoft is encouraging all their users to start using MFA, so the made it free of charge for all the apps of the office 365 suite, including Outlook, Teams, Excel, Word and many more.

The 2 factor authentication can be setup up fairly easily by the end users self. This can be enforced by the administrator by requiring 2 factor authentication. The first time a user logs on, he or she will get a notification message to setup MFA. Or you can redirect your users to the following portal to setup MFA: https://aka.ms/mfasetup
How to setup MFA for your end users?
In the office 365 portal go to the Active Users tab, and go to the Setup multifactor authentication page (see below)
