Unsolved problems with blockage, beam aggregation, and even how you hold the phone. Robert Heath is the man "who wrote the book on millimeter wave." He has a major testbed and lab at the University of Texas. He's an enthusiastic supporter, but also a realist. Millimeter waves sometimes are blocked by a window or wall. They definitely won't get to the other side of a building or bounce around most obstacles.
Most of these problems can be solved. Beamforming and beamtracking have great promise. Millimeter antennas can be very small, allowing large arrays that don't take up much space or power. Receivers will often be able to connect to multiple cells; research on how to combine them efficiently is a hot topic. When mobiles are part of the network, everything will become much harder. Take a look at the slides below.
Turn around and your body may affect the signal.