We present a fast gradient domain video processing using hierarchical data structure which subdivides the processing region into an octree data. It is hard to handle large video processing by solving a 3D Poisson equation, as the derived linear system is usually large. Solving the system requires large memory space and long computational time, which makes it intractable on a standard computer. To address the scalability problem, rather than processing the video in the gradient-domain pixel by pixel, we perform the video processing in a reduced space using octree data structure, which significantly reduces the variables. We show that the proposed octree approach is efficient in both seamless and mixing gradient-domain video processing. The method enables to perform video processing in greatly reduced computational time and memory space, while receiving visually identical results with that computed from the full solution.