How ManiaGAI works

DanceGAI is the team behind ManiaGAI, an AI osu! mania beatmap generator. DanceGAI is building AI infrastructure that turns music into playable experiences, and ManiaGAI is its first product: you upload a song, and it generates a complete, playable osu! mania mapset — timing, notes and a spread of difficulties — packaged as a standard .osz. The song can be an uploaded audio file or the song of an osu! beatmapset you link to; either way only the audio is used, never the existing charts. Version 2 of the system is live on this site. osu! mania is the first output format: the system is built for vertical scrolling rhythm games (VSRG) in general.

This page explains how it works and why it is designed the way it is. We are open about the architecture and the engineering decisions; we don't publish the details someone would need to rebuild it. Where a number isn't public yet, the page says so rather than guessing.

Technical summary

A beatmap is not a transcription

The obvious way to frame automatic beatmap generation is “audio in, neural network, notes out”. In practice that framing produces charts that technically follow the music and are unpleasant to play. A good osu! mania map is closer to an arrangement than a transcription: a mapper decides which sounds deserve notes, how dense a section should feel, where to put a jack or a long note, how patterns flow between hands, and how all of that changes between an Easy and an Expert difficulty.

So ManiaGAI treats beatmap generation as a sequence of distinct problems — understanding the music, planning the gameplay, placing the notes, and checking the result:

1. Music
2. Understanding
3. Planning
4. Generation
5. Refinement
6. Beatmap

1. Music understanding

Everything downstream depends on a precise picture of the song. The first stage turns raw audio into two things: a timing grid, and a high-resolution musical representation of every moment of the song.

Timing. Beat and downbeat activations come from Beat This, an open-source beat-tracking model. Turning those activations into an osu! timing grid — BPM and offset, including songs whose tempo changes — is our own grid-fitting method, and it produces the same family of grids the generation models were trained on. Timing is the part players notice first: a chart a few milliseconds off the beat is unplayable no matter how good its patterns are, so it is treated as a first-class problem, not a preprocessing step.

Musical features. The audio is encoded with MERT-v1-95M, a self-supervised music representation model, and aligned to the timing grid — so the models see the music beat by beat: onsets, texture, intensity and how they change across the song. Silent stretches are detected separately, so nothing is charted over them.

2. Beatmap planning

The central design choice in ManiaGAI is to separate what should happen from how notes are placed.

A dedicated planning model reads the musical representation and decides the gameplay structure of the whole song before any note exists: how dense each section should be, which kinds of patterns belong where, how the difficulty should rise and fall, and how all of that fits the requested difficulty and style. Its output is a plan that describes the chart section by section — local difficulty and the mix of long notes, jacks and other pattern types — not notes.

The planner is a Gaussian diffusion model (about 6M parameters). It generates the plan for every section of the song jointly, conditioned on the music, the key count and the target difficulty and style. Diffusion suits this job: a plan is a smooth, continuous description of a song, and generating it as a whole lets the planner balance the entire song at once.

Planning at this level gives the system something a note-by-note model lacks: a view of the whole song. A chorus can be planned to hit harder than the verse before it; a breakdown can be planned to breathe.

3. Note generation

A second model turns the plan into actual note placement — which columns, which notes are taps and which are long notes, how patterns connect. Planning determines the structure; note generation turns that structure into playable notes that follow both the plan and the audio.

The note model is a MaskGIT-style non-autoregressive transformer (about 51M parameters). Charts are represented as discrete pattern tokens, and instead of writing them one at a time from start to finish, the model fills in the whole chart and then refines it over several passes — committing the notes it is confident about and revisiting the rest. Every pass sees the context on both sides of every section, which is what keeps patterns consistent across a song. How those passes are scheduled across a song is our own refinement method.

Why ManiaGAI uses a Planner + Noter →

4. Controllable generation

ManiaGAI is not a “generate and hope” system. The same song can be generated under explicit controls, and the controls are part of what the models were trained to follow:

• Key count — 4K to 9K.
• Difficulty — any selection of star ratings from ★1 to ★10. One song produces a whole mapset, Easy through Expert, in a single run.
• Style — gameplay characteristics such as the share of long notes and how much jack, inverse-LN, short-LN and delay patterning a chart contains, each adjustable on the generator.
• Density — follows from the target difficulty and the plan, rather than from a fixed rule.

Every difficulty is generated from the same understanding of the song, so a mapset reads as one family of charts rather than unrelated ones. Next, we are working toward learned style representations — describing a mapping style by example rather than by a handful of sliders.

5. Validation and quality control

Before a chart reaches you, the system checks it:

• Timing — the written timing points are checked against the detected grid.
• Invalid and unplayable notes — overlapping notes, broken long notes and placements the game can't represent are resolved, and nothing is charted over obviously silent audio.
• Measured difficulty — every chart's star rating is measured with the same star-rating algorithm osu! uses, and the difficulty is named after the measured value, not the requested one.
• Consistency across the mapset — a chart that comes out easier than the one meant to sit below it, or nearly identical in difficulty to a neighbour, is dropped rather than shipped. On the generator, you see which stars were dropped and why.
• Output validity — the result is packaged as a standard .osz with audio, background and hitsounds that imports into osu! directly.

By the numbers

Songs in the training set

86,255

Training charts

150,622

Mappers

~14,500

Unique audio

4,227 h

Training features on disk

~0.7 TB

Training compute (logged, production lineage)

~580 GPU-hours

Final note-model run

~4 days

Model parameters

~57M (51M notes + 6M planner)

Full mapset generation

~20 s

Key modes

4K–9K

Difficulty range

★1–★10

Difficulties per run

up to 10 (fewer for long songs)

Training data

The production models learn from 150,622 human-made osu! mania charts across 86,255 songs — 4,227 hours of unique audio — by roughly 14,500 mappers. Most of the work in a system like this is not the model; it is the data: deciding what counts as a chart worth learning from, making sure the timing of every training example is right, and representing charts in a form that captures what makes them playable.

Explore the training dataset →

Evaluation: timing accuracy

We benchmarked our timing against human timing on every ranked osu! mania mapset with a single, constant tempo that we could evaluate — 2,501 songs — alongside the open-source Mapperatorinator timing mode and ManiaGAI v1, our previous detector. A song counts as correct when the BPM matches the mapper's (allowing for half/double tempo) and the beat phase is within the given tolerance.

Constant-tempo ranked songs only; “one timing line” = the song was timed with a single line, as the mapper did (an extra line on a constant-tempo song is a false tempo change). Measured on a configuration close to, not identical with, the shipped one.

Read the full timing benchmark →

Training

The planner and the note model are trained separately, each on the representation it works with — the planner as a Gaussian diffusion model over a Gaussianised description of each section's gameplay, the note model by masked-token prediction over pattern tokens. Training samples are balanced across key counts and difficulty levels so that 8K and ★10 charts are not drowned out by 4K and ★4, and the note model trains in mixed precision.

Every change is judged on the generated charts themselves — timing accuracy, measured difficulty against the target, playability — not on training loss alone.

Inference and performance

A useful AI system is not only about model quality; it also has to be fast enough to use. A ManiaGAI request runs like this:

1. In your browser: the song is decoded, loudness-normalised and re-encoded as soon as you pick the file, while you are still choosing settings — so the GPU never waits on audio processing.
2. Music understanding on the GPU: Beat This and MERT, with the grid fitting running on the CPU concurrently with MERT.
3. Planning and note generation for every requested difficulty in one batched pass.
4. Validation — star rating measurement, consistency checks.
5. .osz packaging, then the download.

GPU workers start from a pre-loaded snapshot, so even a cold worker is ready in seconds rather than spending that time loading models. A full mapset takes about 20 seconds of GPU compute time for a typical song — not counting upload, queueing or download; one measured 2:50 song with five difficulties took 12.7 seconds on an NVIDIA T4 — timing and music features 3.8 s (run concurrently), planning 1.0 s, note generation for all five difficulties 6.1 s.

How we made generation fast →

What we don't publish

We publish the architecture principles, the engineering decisions behind them, and what the system does and doesn't do — including the open models we build on (Beat This, MERT) and the families our own models belong to (Gaussian diffusion, MaskGIT). We don't publish the methods we developed ourselves or the details that would let someone reproduce the system: how the timing grid is fitted and its tempo segments merged, how the note model's refinement passes are scheduled across a song, how the models are conditioned, exact architectures, losses and hyperparameters, and how the training data is sourced, filtered and constructed. That is the line between being transparent about what we built and handing over the recipe. When we make claims about quality, we intend to back them with published benchmarks — not adjectives.

What's next

Beyond osu! mania. ManiaGAI is designed around a gameplay representation that is not specific to osu! mania: notes and long notes in columns, placed on a timing grid and conditioned on a target difficulty. osu! mania is the first output format, not the boundary of the system.

• #osumania
• #osu
• #VSRG
• #RhythmGames
• #AIBeatmapGenerator
• #BeatmapGeneration
• #RhythmGameAI
• #AIMusicToGame
• #MaskGIT
• #DiffusionModels
• #MERT
• #BeatThis
• #ManiaGAI
• #DanceGAI