I know polar fleece may seem too thick for quilting, but I think it works. It is thick but it's lightweight while still being very warm and cozy. Probably NOT a good quilt for Florida, but fine for Wisconsin or snowy New York.
I haven't tried it often, although the quilt I made with it became one of my favorites. I used it not only as the backing, but also in the pieced top. This one went to my brother-in-law Tom and if I remember right, it was for his 50th birthday. This year he said he turned 12 on his birthday so I must be wrong about the 50th! Regression isn't a bad thing and I do believe you should get to be any age you choose.
A quilting friend just asked about using flannel as a backing and I'm sure that's why I was thinking about the polar fleece. I loved the feel of it and the distinct detail of the quilting. No batting was used. I only tried flannel once (Tavern quilt); I like the results and will probably do another. I've never used wool although I've always planned to. Plaid woolens in a simple, classic pattern like Ohio Star would be nice. Old flannel shirts, cut into 4" squares with a lightweight denim backing would be interesting too. Ah... so many quilt ideas!