How to -correctly- use SharePoint Property Bags

There are quite a few posts on property bags and how to use them. All of them are incomplete because:

1) There are several ways to access the property bags and the correct one is not always shown, and,
2) The code is incomplete.

Microsoft suggests that methods like SetProperty, AddProperty and DeleteProperty are used to access Property Bags. Also, remember to call SPWeb.Update() to save the changes. If you don't your changes won't persist.