As a one-line summary, that's about right.
You don't really want to be tracking master, so you really need one branch per stable version. You can't have one version which tracks multiple stable branches because sometimes when we backport a patch, it conflicts with the same patch in another branch (e.g. API changes mean things have to be done differently).
At Lancaster, and Moodle HQ, we have one branch per (non-core) plugin too, but you may not need this added complication. It worked well for us, but an upgrade event probably took a day of preparation, planning, and git hyjinx. That said, a day spent doing it well once, is a drop in the ocean compared with dealing with merge conflicts, broken files, broken plugins, etc. if we were to do it wrong. This method makes it much harder to make mistakes.
Another note that's worth mentioning is that you need to be careful with git pull - use git fetch instead. By default, git pull is the same as doing git fetch, followed by an implicit git merge which you do not want. You ideally want to merge a tag, not a branch.
Good luck, and please feel free to ask any more questions or for further clarifications.
I'll try and get David Mudrak to write a blog post now that he's been trying this method for a few months with any tips, tricks, gotchas, etc.
Andrew