Woven hat with band