Out of curiosity, did you take any other actions besides moving after casting?
It does require that a caster maintains "concentration", similar to the Black Blade of Disaster. If the caster attacks, casts another spell or does anything more taxing than walking around, the spell is broken.