martian garden.draft04 / How to Make a Music Video with Midjourney

This is an original and (unfinished) song made in Ableton with classic production techniques.

The video portion is fully generative AI made with Midjourney as the primary tool.

Here is the process for how I made the video:
Midjourney allows you to generate still images with one prompt and then animate them into a 5 second animated clip. The final cut has over 30 animated clips, but the process was a deep dive: I generated over 200 still images and nearly 100 video clips. You could be a lot more efficient but this was my first time doing something like this so lots of mistakes were made along the way.

Here’s a detailed breakdown of the workflow for anyone interested:

The Toolkit:
Midjourney ($30 Plan): For all still image and video generation. I think I used most of that $30 for the project. You could be a lot more efficient as I was totally learning it for the first time.
Gemini Pro 2.5: I used a Canvas document as a production bible for brainstorming the story, organizing all my reference image links, and iterating on hundreds of prompts in one place.
DaVinci Resolve (Free Version): For the final video edit, color grading, and assembly.

The Creative Process (Step-by-Step):
Character Creation: The story is about two sci-fi "supervillains for good," so I needed consistent characters. I used reference photos of my friends and the --cref parameter to create AI-generated portraits of them that I could use consistently throughout the project.

World-Building: I established the aesthetic—a "hyperrealistic, brutalist tropical" sci-fi world—by generating a bunch of "hero" shots of the key locations, like the estate and the club. This gave me a library of style references to pull from.

Scene Generation: This was the key to cohesion. I used a combination of Character References (--cref) for my friends and Style References (--sref) for the world shots, all in the same prompt. This allowed me to place my established characters into the specific worlds I had already created, ensuring the lighting and style were always consistent.

Animation: All animation was done using Midjourney in browser on the final, upscaled still images. I learned that for the best quality, you have to upscale your still image to the highest resolution before you generate the video from it. For the best results, generate your videos in raw hd and make sure to add text to the prompt defining the movement you want (ie pan, move over crowd, dive)

Final Edit: All the generated video clips were imported into DaVinci Resolve, where I cut them together to the music. I added a film grain overlay from a free sample pack I found online.


A Few Tips I Learned the Hard Way:
Quality First: upscale your still images to the highest quality possible before you even think about animating. It makes a huge difference in the final output. I had to redo the first 30 seconds because of this.

Guiding the Camera: Always use manual animation and you can strongly influence the motion by describing the camera movement in the text prompt (e.g., "a slow, cinematic pan right"). You can also affect the environment but should aim to have the scene accurately styled at the image stage.

Be Efficient: You'd be surprised how often the first or second generation is the one you need. Trust your eye, and save your fast hours for the shots that really matter.


The Music
I started to work on this song recently and as a novice producer, this project was a huge learning curve. Once I got to this version of the song I started to send it around to friends and got varied feedback. But, I felt like I envisioned a world that could better frame the music and I think it does make the whole thing more enjoyable for everyone.

Happy to answer any questions