Games from the Microsoft Store won't necessarily run offline without some prior action on your part. The Microsoft Store requires you to nominate a single device as your "offline" device, which can be used to run apps and games with restricted licenses without an Internet connection.
Unless you've set a device to be your offline device, you may find you can't launch games until you're back online. Since only one device can be used offline at any time, you'll need to think carefully about which of your Windows products should be used for mobile gaming. You can't just change the setting when you switch device either – Microsoft allows only three changes in each calendar year.
The easiest way to assign an offline device is through the Microsoft Store itself. Launch the Store app on the device you want to use offline. When it opens, click the three dots menu button ("…") in the top-right corner, and then click "Settings."
Scroll down to the "Offline Permissions" heading and turn the toggle button to the "On" permission. You'll see a warning notifying you of how many offline device changes you have remaining. Once you confirm the prompt, your current device will become your offline device – if you'd previously assigned this status to another PC, it will now be revoked and you won't be able to play games offline.
Not every game is affected by this setting. It typically applies to games you've purchased which identify as PC or Xbox titles, not the simpler mobile-style games which can also be found in the Store.
To make sure you can play your favourite titles offline, you should launch them once while connected to the Internet. This will ensure the necessary licensing information is available on your device for when you go offline. You should then be able to run the game at any time, with or without a connection, and enjoy gaming on the go.