This is just to say that the topics of hyperspace and XDs have inspired generations of physicists. And whoever it was who first did the calculation that shows string theory needs extra dimensions to make sense, it must have been one of the most exciting moments I can imagine for a theoretical physicist.
But XDs have come a long way, and were around long before string theory. People sometimes ask me why my talks never mention the earlier works on the topic. The reason is that the theories with XDs proposed in the 1920ies by Theodor Kaluza and Otto Klein, are in their idea different to the 'modern' XDs. Yet, this usually takes too much time to clarify in a talk, so I rather skip it. However, since you - and yes, I mean YOU who you are just raising your eyebrows - are of course the most attentive reader there is, I want to elaborate somewhat on these 'early' XDs since I noticed very little people actually read the original works by Kaluza and Klein.
The first mentioning of adding another dimensions to our three space-like dimensions that we experience every day goes to my knowledge back to Nordström in 1913 . He however did not yet use General Relativity (GR) to build his theory upon. Since we know today that the gravitational potential is not a scalar field, but described by the curvature of space time, let us skip to the next attempt which uses GR as we know it today.
GR couples the metric tensor (g) to a source term of matter fields, whose characteristics are encoded in the stress-energy tensor of the matter. All kind of energy and matter results in such a source term, and hence causes the metric to deviate from flat space. This theory does not say anything about the origin of the source terms. The matter and its properties have to be described by another theory - for example by electrodynamics. Electrodynamics on the other hand has a similar problem. The source for the electromagnetic field (charged particles) is not described by Maxwell's equations . They need to be completed by further equations, e.g. the Dirac equations.
In the beginning of the last century, physicists had just understood gravity as a geometrical effect instead of a field in Minkowski space, so it was only natural to try the same for other fields as well, with the obvious next choice being the electric field. The idea of the early XDs is plain and simple. Einstein's field equations are a set of non-linear differential equations for the metric tensor. They are built up of the Ricci-tensor (two indices) which is a contraction over the full curvature tensor (four indices), and the curvature scalar - a further contraction of the Ricci-tensor. Such a contraction is basically a sum over two indices. The indices on these tensors label space-time directions - that is, in the standard case of GR with three space and one time dimensions, they run from 1 to 4 (or, depending on taste, sometimes from 0 to 3).
Now if one had an additional dimension, then two things happen with Einstein's field equation. First, one has more equations because there are more free indices. Since the Ricci tensor and the metric are symmetric, the number of independent equations is D(D+1)/2, here D is the total number of dimensions. The second thing happening is that the equations with the indices belonging to the 'usual' directions acquire additional terms since the sum runs over the additional indices as well. The trick is then to separate the usual part (sum from 1 to 4) from the additional part (sum over the extra dimension), shift the additional part to the other side of the equations, and read it as a source term. In such a way, one obtains a source term even if the higher dimensional field equations were source free.
Kaluza and Klein
The result is that components of the higher-dimensional metric tensor appear as source terms for the four-dimensional sub-sector that we observe. The first such approach was Theodor Kaluza's whose ansatz uses one additional dimension. In the remaining entries of the metric tensor (those with one index being a 5) he put the electromagnetic potential with a coupling constant alpha (since the metric tensor is dimensionless but the electromagnetic potential isn't)
(Here, the large Latin indices run over all dimensions, the small Greek indices over the usual four dimensions). Kaluza apparently sent a draft of his paper to Einstein in 1919, to ask for his opinion. It got published with a delay of two years .
Kaluza derived the higher dimensional field equations in the linear approximation. Generically, all the components of the metric tensor will be functions of all coordinates, including the additional one. This however is in conflict with what we observe. Kaluza therefore added what he called the 'cylinder condition' that set derivatives with respect to the additional coordinates to zero. In the linear approximation, he then found the ansatz to reproduce GR plus electrodynamics.
However, the use of this linear approximation is not necessary, as was shown by Oskar Klein five years later . Klein used a different ansatz for the metric which has an additional quadratic term:
(Sorry, coupling constant is missing, my fault not Oskar's) And he assumed the additional coordinate is compactified on a circle. Then, one can expand all components in a Fourier-series and the zero mode will fulfill Kaluza's cylinder condition' that is, it is independent of the fifth coordinate. However, if you compare both ansätze , Klein's and Kaluza's, you will notice that Klein set the g55 component to be constant to one. This is an additional constraint that generally will not be fulfilled. In fact, the additional entry behaves like a scalar field and describes something like the radius of the XD. At this time however, people had little for additional scalar fields.
Klein's derivation is simply one of the most beautiful calculations I know. One just writes down the higher dimensional field equations, parametrizes the metric tensor according to Klein's ansatz, decomposes the equations - and what comes out is GR in four dimensions (in the Lagrangian formulation as well as the field equations), plus the free Maxwell equations.
(Here, the supscript (4) and (5) refer to the 4 and 5 dimensional part of the curvature/metric). Further, the geodesic equation gets an additional term which is just the Lorentz force term and thus describes a charged particle moving in a curved space with an electromagnetic field.
In the course of this derivation, one is lead to identify the momentum in the direction of the fifth coordinate as the ratio of charge over mass (q/m). It can be shown that this quantity is conserved as it should be. Klein concluded that this charge is quantized in discrete steps (this is a geometrical quantization), the first example of the Kaluza-Klein tower.
Extensions and Problems
To understand the excitement this derivation must have caused one has to keep in mind that this was 30 years before Yang and Mills, and the understanding of gauge theory was not on today's status. With today's knowledge, the argumentation appears somewhat trivial. One adds an additional dimension with U(1) symmetry, the compactified dimension. The resulting theory needs to show this symmetry that we know belongs to electrodynamics. From this point of view, it is only consequential to extend the Kaluza-Klein (KK) approach to other gauge symmetries, i.e. non-abelian groups. This was done in 1968 .
One has to note however that for non-abelian groups the curvature of the additional dimensions will not vanish, thus flat space is no longer a solution to the field equations. However, it turns out that the number of additional dimensions one needs for the gauge symmetries of the Standard Model U(1)xSU(2)xSU(3) is 1+2+4=7 . Thus, together with our usual four dimensions, the total number of dimensions is D=11. Now exponentiate this finding by the fact that 11 is the favourite number for those working on supergravity, and you'll understand why KK was dealt as a hot canditate for unification.
But there are several problems with the traditional KK approach. First, meanwhile the age of quantum field theory had begun, and all these considerations have been purely classical and unquantized. Even more importantly, there are no fermions is this description - note that we have only talked about the free Maxwell equations. The reason is easy to see: fermions are spin 1/2 fields and unlike vector bosons one can not just write them into the metric tensor. One can of course add additional source terms, but this makes the idea somewhat less appealing . The high hope had been to explain all matter and fields from a purely geometric approach.
If one thinks more about the fermions, one notices another problem: right- and left-handed fermions belong to different electroweak representations, a feature that is hard to include in a geometrical interpretation. Furthermore, there is the problem of stabilization of the compact extra dimensions (the sizes should not or only negligibly depend on the time-like coordinate), and the problem of singularity formation from GR persists in this approach. However. If I consider what landscape of problems other theories suffer from, it makes me wonder why the KK approach was so suddenly given up in the early 70ies. A big part of the reason might simply have been that the quark model got established, and it was the dawn of the particle-physics era.
The 'modern' extra dimensions differ from the KK approach by not attempting to explain the other standard model fields as components of the metric. Instead, fermionic- and gauge-fields are additional fields that are coupled to the metric. They are allowed to propagate into the extra dimensions, but are not themselves geometrical objects. Most features of the KK approach remain, most notably the geometrical quantization of the momenta into the extra dimensions and thus the KK-tower of excitations. So remains the problem of stabilization, singularities and quantization (for higher dimensional quantum field theories the coupling constants become dimensionful). However, for me this 'modern' approach is considerably less appealing as one has lost the possibility to describe gauge symmetries and standard model charges as arising from the same principle as GR.
But obviously, the largest problem with the KK approaches was - and still is - that it is not clear whether it is just a mathematical possibility or indeed a description of reality. As Oskar Klein put it in 1926:
- "Ob hinter diesen Andeutungen von Möglichkeiten etwas Wirkliches besteht, muss natürlich die Zukunft entscheiden."
- "Whether these indications of possibilities are built on reality has of course to be decided by the future." 
 And if you read the wikipedia entry on Perry Rhodan you find a use for the word 'Zeitgeist'...
 G. Nordström, "Zur Theorie der Gravitation vom Standpunkt des Relativitätsprinzips" Annalen der Physik, vol. 347, Issue 13, pp.533-554 (1913); G. Nordström, "Über die Möglichkeit, das elektromagnetische Feld und das Gravitationsfeld zu vereinigen" ( "About the possibility to unify the electric field and the gravitational field" ) Physik. Zs. 15, 504-506 (1914) [Abstract]
 T. Kaluza, "Zum Unitätsproblem der Physik'' ("On the Problem of Unity in Physics, Sitzungsber. Preuss. Akad. Wiss. Berlin (Math. Phys.) (1921) 966.
 O. Klein, "Quantentheorie und fünfdimensionale Relativitätstheory" ("Quantum Theory and fivedimensional General Relativity", Z. Phys. 37, 895 (1926).
 Unlike to what the Wikipedia entry states, the Lorentz force law can not be derived from the Maxwell equations without further assumptions (like a Lagrangian for the coupled sources). E.g. Maxwell's equations are perfectly consistent for a static superposition of two negativley charged objects, just that we know the charged particles would repel and the configuration can't be static.
 R. Kerner, "Generalization of the Kaluza-Klein theory for an arbitrary non-abelian gauge group", Ann. Inst. Henri Poincare, 9, 143-152 (1968)
 Contrary to the wide spread believe, the plural of the German word 'Ansatz' is not 'Ansatzes' but 'Ansätze' (pronounced 'unsetze'). 'Ansatz' could be roughly translated as 'a good point to start', or a preparation. E.g. the pre-stage for yeast dough is called 'Ansatz'...
 Which finally brings us to the topic on which I lost two years during my Ph.D. time, namely the question whether one can built up the metric tensor from spin 1/2 fields. I only learned considerably later that most of this approach had been worked out in the mid 1980ies, see e.g. hep-th/0307109 and references therein.
 He indeed writes it has to be decided 'by' the future not 'in' the future. Quotation from Ref. 
- Physics Notes on Kaluza Klein theories by Viktor Toth
- Overduin and Wesson, Kaluza Klein Theory, gr-qc/9805018
- Van Dongen, Einstein and the Kaluza Klein particle, gr-qc/0009087
- Wikepedia on Kaluza-Klein theories
- Jim Adson, Kaluza, Klein, and the Grand Unified Theory
- Häggblad, Kaluza Klein Theory
- Van de Schaar, Kaluza Klein Theory
- H. Goenner, Kaluza's five dimensional unification (from this living review)