After the wars of Julius Caesar against the tribes of northern Gaul and what is now Belgium, the Romans created two provinces in Germania: Germania Inferior (closest to the North sea) and Germania Superior (farther inland), both provinces carved out of what was once Gaul and situated east of the Rhine River.

The Emperor Augustus had originally planned, after the wars along the Rhine, to create a single province of Germania Magna, with its eastern border on the Elbe, but with its capital at Cologne (Colonia Agrippinensis). Following the failure of this project and after the notorious defeat of the Roman General Varus, the Rhineland became a military zone, garrisoned by two armies of four legions, each under a senatorial legate. However, perhaps to obscure defeat and maintain a claim to Free Germany, the forces on the northern Rhine were still known as the army of "lower" (Inferior)Germany, and troops on the southern Rhine, the army of "Upper" Germany (Germania Superior).

The areas the armies controlled were effectively provinces, with their capitals at Cologne and Mainz (Mogontiacum) respectively, but their formal designations as Germania Inferior and Germania Superior did not come until AD 90, as part of the Flavian imperial reorganization of the Rhine frontier. During the high empire, the German provinces flourished, thanks to imperial military spending and trade with the empire and those parts of the far north not conquered: under Diocletian, Germania Inferior was renamed Germania Secunda, while Superior became Prima and Maxima Sequanorum, all three belonging to the Diocese of the Gauls. During the fourth century, the two Germanies were strongly defended by the imperial armies but were badly damaged by increasing incursions of Germanic Frankish and Alamannic attacks. By the mid-fifth century, their connection with the western Empire had largely disappearead.

For more information, see : Germania Inferiorand Germania