Sounds like a pretty good plan. Pooping the hot air out and bringing in fresh right close to the torch is The way to preserve your indoor climate control.
Im not sure how much fire power you have running but a 6 inch is pretty nominal. 8 or 10 inch will move more air.
I work borosilicate on a GTT Mirage so I lean towards a 12 inch minimum. I currently run a 1,600CFM 15 inch fan.
Im also not much on running ducting, my fan immediately shoots the dirty air outside.
Here is a video of my ventilation test with 3 smoke bombs.
https://www.youtube.com/watch?v=zwWqj_HuDsg