Early Quantum Physics
The Quantum Theory
- Setting the Scene 1.1 The Classical Period Modeling the Atom: 1808 - 1909
John Dalton (1766 - 1844) built on the work of his predecessors (particularly Lavoisier and Proust) to create what he called A New System of Chemical Philosophy (1808). His theory proposed that all matter consists of indivisible atoms, and for any particular element, all of its atoms exhibit identical properties. Furthermore, chemical reactions entail the re-arrangement of atoms in fixed, whole-number ratios. This was a model of the atom introduced specifically to account for the changes in the quantities of reactants and products during a chemical reaction. Dalton’s atoms were static in the sense that they had no dynamics attached. They were discrete, indivisible, conserved units with a fixed mass, and the only "physics" in the theory was combinatorial – i.e. they combine in whole-number ratios and are neither created nor destroyed in reactions. The theory works perfectly well as a counting rule – it doesn’t need to say anything about what an atom looks like or how it moves in order to explain its definite and multiple proportions.
Around 1810 Dalton had his friend Peter Ewart build a set of wooden ball models drilled with holes, so they could be pegged together to demonstrate compound formation. The “visual aid” of little solid balls was therefore genuinely Dalton's idea and tells us why his model eventually came to be known as the “billiard ball model,” but this name leans heavily on the subsequent work of James Clerk Maxwell. In the 1860s Maxwell showed that Dalton’s atoms were more than just “units of account” for chemistry – he proved that atoms are dynamic particles with measurable kinetic energy and reliable statistics. Over the ensuing three decades Ludwig Boltzmann generalized Maxwell’s work to create statistical mechanics, bridging the world of individual atoms and the principles of thermodynamics.
In 1814 Joseph von Fraunhofer mapped over 500 dark lines in the solar spectrum, but he had no idea what caused them. It was not until 1859/1860 that Gustav Kirchhoff and Robert Bunsen showed that each chemical element, when heated to incandescence, produces its own characteristic set of bright emission lines. Passing light through a cool gaseous element showed that the gas absorbs light at precisely the same wavelengths as its hot variant radiates them. This led to the discovery of the previously unknown elements caesium and rubidium, and in 1868 to the discovered helium by the analysis of sunlight. Spectral lines were then associated with chemical elements without any notion of what aspect of the element might be responsible for producing them, but some physicists (Stokes, Maxwell) had already begun to suspect that the lines were telling them something about the internal structure of the atom itself. This suspicion was reinforced by the discovery of regular patterns in the line sequences, like the Balmer series found in hydrogen (1885 and later generalized by Rydberg in 1888/1890), and like the Ritz combination rule. Walter Ritz recognized (1908) that the frequencies of any element's spectral lines are always the sum or difference of two other line frequencies, strongly indicating that these lines correspond to differences between a set of internal energy states.
In 1897 J.J. Thompson discovered the electron, rendering Dalton’s billiard-ball model of the atom no longer tenable. Thompson proposed replacing it with his "plum pudding" model of the atom (1904) in order to maintain the electrical neutrality of the atom, but when Ernest Rutherford discovered that the positive charge and most of the mass of the atom resided in an extremely small nucleus (1909), this in turn made Thompson’s plum pudding model untenable. Atomic electrons must remain outside the nucleus, but there was still no clear account of their arrangement or motion. Nonetheless, this was very suggestive of the familiar planetary model of the solar system, and this analogy was therefore an obvious contender for investigation. If electrons were to be conceived as orbiting the nucleus on the model of planets orbiting the sun, then the nature of the orbits was unclear. Furthermore, neither Thompson's plum pudding model nor Rutherford's nucleus model could account for the line structure in atomic emission and absorption spectra that had been discovered by Fraunhofer.
Joseph Fourier had shown how any kind of periodic motion could be decomposed into sinusoidal components consisting of the fundamental frequency and a series of harmonics (integer multiples of the fundamental, identified by their harmonic number ‘n’). Maxwell had then shown how each of the Fourier components describing an accelerating charge would generate electromagnetic radiation. It was then entirely reasonable to make a connection between the complicated collection of line frequencies in an atomic spectrum and the Fourier components of some complicated periodic electron orbit, but there were problems with this line of inquiry.
Firstly, no convincing classical atomic orbit could be found that actually reproduced all of the observed spectral regularities – the correspondence between the Fourier components and the lines in atomic spectra was found to hold only for components with large values of the harmonic number ‘n.’ Even if such an orbit were ever to be found, Maxwell’s electrodynamics predicted that an electron would still radiate away its energy and fall into the nucleus in an absurdly short time. Moreover, its radiation should change continuously as the orbit shrinks. This was inconsistent with the empirical data – atoms are stable, and their spectra consist of sharp, discrete lines. By 1913, the empirical situation was clear – line spectra are atomic in origin, the lines appear in series governed by integer quantum numbers, and the frequencies are differences between a discrete set of quantities – but there was still no theory available to account for these observations.
The Nature of Light: 1672 - 1905
In 1672, Isaac Newton published his landmark paper demonstrating that white light is composed of distinct colored rays. While Newton privately favored a corpuscular (i.e. particle) view of light, Christiaan Huygens and Robert Hooke separately challenged Newton’s interpretations, preferring a wave-like view of light. Huygens developed his pulse/wave theory in 1678 (publishing it in 1690 as the Traité de la Lumière), using longitudinal wave fronts to explain reflection and refraction. In 1704 Newton formally published his comprehensive corpuscular model in his Opticks, arguing that his hypothesis provided the best explanation of light's straight-line motion and the absence of shadow bending. Newton's authority meant that his corpuscular theory dominated for almost a century until, in 1801, Thomas Young demonstrated interference with his famous double-slit experiments. Together with Augustin-Jean Fresnel’s work in the 1810s, Newton’s corpuscular hypothesis gave way to the wave theory of light, which held sway for another century.
During the late 1800s, physicists were deeply interested in the spectrum of thermal radiation emitted by a "black body" – an idealized physical body that absorbs all incoming electromagnetic radiation without reflecting any. Because it reflects no light, an unheated black body appears completely black, but when heated it emits light across a spectrum determined entirely by its temperature. The task was to give a mathematical account of this spectrum. In 1896 Wilhelm Wien presented an equation for this spectrum, but it failed to match experimental data at long wavelengths (i.e. in the infrared region). In 1900–1905, Lord Rayleigh and Sir James Jeans derived an equation using strict classical physics. While their equation made correct predictions at the long wavelengths where the Wien equation failed, it failed itself at short wavelengths. The Rayleigh-Jeans equation predicted that the amount of energy emitted by an ideal black body would approach infinity as the wavelength approached zero (i.e. in the ultraviolet region). Later dubbed the "Ultraviolet Catastrophe" by Paul Ehrenfest (1911), this breakdown indicated that classical physics was in need of revision.
In late 1900, the German physicist Max Planck sought an equation that would match the precise experimental data across the entire spectrum. Planck was successful in deriving such an equation, but his derivation entailed a radical assumption: the atomic resonators in the cavity walls could only absorb or emit energy in discrete packets, where the energy of a packet was directly proportional to its frequency (the constant of proportionality now bears his name: Planck’s Constant). Planck initially regarded this ‘quantization’ as an uncomfortable mathematical trick to rescue the thermodynamics of thermal radiation, remaining convinced that light itself propagated as continuous waves.
1.2 The Transition Period
The Nature of Light
In 1905, Albert Einstein took the revolutionary step of applying Planck’s notion of quantization not to the emitting/absorbing surface but to light itself. He proposed that electromagnetic radiation travels through space as localized packets of energy he called ‘quanta,’ later to be called ‘photons’ (Lewis, 1926), and he used this hypothesis to account for the photoelectric effect. The physics community resisted Einstein's “light-quantum” hypothesis for years, but subsequent experiments confirmed that light exhibits both wave-like properties (as shown by interference patterns generated in the two-slit experiment) and particle-like properties (as shown in collision events such as the photoelectric effect), demonstrating that the nature of light exhibits a fundamental dependence on the nature of the measurement apparatus.
Modeling the Atom: 1913 - 1925
In 1913 Neils Bohr broke decisively with classical electrodynamics. He postulated that atoms could exist only in certain allowed “stationary states” – states in which they neither emitted nor absorbed radiation despite the classical expectation that an accelerating electron should radiate continuously. Radiation was instead associated with transitions between these stationary states, the frequency being determined by the energy difference between them (consistent with the Ritz combination rule). Furthermore, Bohr postulated that the lines in atomic spectra were not the result of harmonics but of the difference in electron energy for different ‘orbits.’ So Bohr pictured instead an electron making a transition from its own specific orbit ‘n’ (so ‘n’ is no longer the harmonic number but the rank of the electron orbit, or quantum number) to a different orbit ‘m’ (where n < m for energy absorption, and n > m for energy emission). Note that this implicated Planck’s Constant by virtue of Planck’s relationship between energy and frequency.
Some of what appeared to be individual spectral lines turned out on closer inspection to be clusters of finer lines – the "fine structure" – which Arnold Sommerfeld later addressed (1915/1916). Inspired by the way that Johannes Kepler had introduced elliptical orbits in planetary motion to solve problems in celestial mechanics, and in conjunction with Einstein's relativistic corrections due to the speeds of the electrons, Sommerfeld’s modification to the Bohr model of the atom succeeded in accounting for important aspects of the observed fine structure. The Bohr-Sommerfeld model showed good agreement with the spectroscopy data for the Hydrogen spectrum (the Balmer Series) but not for the atomic spectra of other elements. More interestingly, though, Bohr showed that the Rydberg formula held not just for large orbits (i.e. consistent with a high harmonic number ‘n’ in the classical case) but for all ‘n’ (i.e. for all orbits in the quantum case). Thus the Rydberg formula was regarded as empirical evidence supporting the Bohr-Sommerfeld model of the atom.
The classical harmonic interpretation works at high ‘n’ because neighbouring energy levels become closely spaced, allowing the discrete transitions between levels to approximate to the continuous variation modeled by classical mechanics. In other words, when the levels become densely packed, the mathematical distinction between a discrete quantum description and a continuous classical description becomes less significant. But at low ‘n’ the levels are widely separated so the discrete nature of the quantum transitions becomes evident and the classical harmonic picture breaks down. It is important to note, though, that this is not simply a case of Bohr’s quantum description becoming more classical in the high ‘n’ limit. More specifically, in that limit, a transition between two states whose quantum numbers differ by ‘k’ has a frequency that approaches that of the k-th harmonic of the corresponding classical motion. Thus the classical harmonic number ‘k’ acquires a quantum counterpart in the difference between two quantum numbers.
Formalizing this observation as his Correspondence Principle around 1918, Bohr then used it to fill some of the gaps in his model of the atom. In particular, while classical theory predicts line intensities based on the amplitudes of Fourier components, earlier models of the atom offered no way of calculating them. Bohr used his correspondence principle to bridge this gap by proposing that the probability of a quantum transition (and thus the line intensity) is related to the amplitude of the corresponding classical harmonic component. This provided a semi-classical model that went some way to accounting for observed variations in line intensity, extending the theory’s reach beyond the positions of lines in atomic spectra (line frequencies) alone, but a full mathematical account of line intensities was still lacking.
It is worth emphasising at this point that the classical description had attempted to make an association between emitted/absorbed light having a Fourier component labelled by a single harmonic number ‘k’ (i.e. a spectral line was associated with a single electron orbit), whereas the emerging quantum description associated a spectral line with a transition between two stationary states, ‘n’ and ‘m’. The observable frequency was therefore no longer naturally attached to the periodic motion of a single electron orbit, but to the relation between two distinct stationary states. This shift in the interpretation of physical quantities was to have profound implications for the development of the theory. Bohr’s correspondence principle provided valuable “semiclassical rules” connecting classical Fourier amplitudes with quantum transition intensities, but these rules were not derived from coherent quantum dynamics. Thus this semi-empirical interpretation (which came to be known as the “old quantum theory”) increasingly became a collection of successful prescriptions without an underlying mechanics capable of mathematically representing atomic processes.
For example, although Bohr's energy-level picture gave the Ritz combination principle a natural interpretation, the theory still lacked a coherent mathematical framework for deriving the full structure of atomic spectra. Additionally, the Zeeman effect (discovered by Pieter Zeeman in 1896) showed that atomic spectral lines split in a magnetic field. Some aspects could be understood in terms of orbital angular momentum, but there was a residue (the so-called anomalous Zeeman effect) that resisted explanation within the Bohr–Sommerfeld framework. More significantly for future developments, a mathematical account of the differences in line intensities was not available. So Bohr’s initial contribution was to abandon attempts to make classical electrodynamics work inside the atom. He retained the language of orbits, but imposed quantum rules upon them that are fundamentally alien to classical mechanics.
- Heisenberg’s ‘Reinterpretation’
2.1 Matrix Mechanics
As the “old quantum theory” became increasingly unsustainable, it became increasingly clear that classical mechanics was unsuited to describing activity inside the atom. It was being used to describe activity that could not itself be observed (conjectured electron orbits), and it didn’t really work. This raised the question of what physical quantities a genuine quantum mechanics should describe – i.e. what happens if we retain the mathematical relationships between the observable quantities but abandon the unobservable classical orbit altogether?
Whilst working at the Bohr Institute in Copenhagen in 1925, Werner Heisenberg studied optical dispersion theory and the theory of anharmonic oscillators. This research provided Heisenberg with the specific mathematical tools and the necessary conceptual framework for an investigation into the differences in line intensities that had been inadequately addressed by the semi-classical Bohr-Sommerfeld model of the atom. Heisenberg was strongly influenced by the positivism of Ernst Mach, who believed that “science should only deal with observable quantities.” In his seminal 1925 paper (the Umdeutung paper, meaning “reinterpretation”), Heisenberg argued that since no experiment could track the electron's path inside an atom, the concept of an "electron orbit" had no physical meaning and should be discarded. He discarded the idea of an unobservable orbit in favor of the only actually observable data available – the frequencies and intensities of spectral lines. As Heisenberg later stated, "My entire meager efforts go toward killing off and suitably replacing the concept of the orbital paths that one cannot observe."
Bohr had already uncoupled the observable line frequency from a single classical periodic motion, and coupled it instead to a transition between two distinct stationary states. Heisenberg could see that he needed to construct a set of tables, one for each of the numerous variables involved in a transition (e.g. a table for electron positions, a table for electron momenta, etc., later to be identified as mathematical operators). He called the table entries transition amplitudes. Each table entry was uniquely identified by its quantum numbers – the initial state (‘n’) and the final state (‘m’). These quantum numbers, then, identified the energy difference between the two states, and consequently the frequency of the spectral line corresponding to the table entry. The table entries themselves (the probability amplitudes) were related to the probability of the transition, and consequently to the intensity of the spectral line.
To construct his theory, Heisenberg retained Bohr’s concept of stationary states, but he fundamentally reinterpreted the role of classical harmonics. Rather than describing single unobservable electron orbits, they became his transition amplitudes – data concerning transitions between distinct stationary states. Heisenberg called these transition amplitudes “virtual harmonics,” retaining a superficial connection with classical theory but shifting the meaning of "harmonics" from the motion within an inferred orbit to the amplitude of a quantum transition.
When considering electromagnetic radiation in Heisenberg’s scheme, the position table is the principal operator – others (e.g. the momentum table) can safely be sidelined. An electron can no longer be assigned a precise classical position in its “assumed trajectory,” so the entries in Heisenberg’s position table do not denote position coordinates at a specific instant. The position of an electron can only be given by a probability distribution, so squaring the probability amplitude does not give the probability of the transition from one precise location to another – rather it gives the probability of the transition from one position distribution to another. An entry in the position table says how strongly two stationary states are connected.
The position table, denoted ‘X,’ contains entries X(n,m) representing the transition amplitudes between stationary states ‘n’ and ‘m’. The values of ‘n’ and ‘m’ identify the observable frequency of the spectral line of emitted or absorbed energy during that transition. A given entry in a table quantifies the strength of the coupling between these two states. Ultimately, Heisenberg replaced spatial trajectories (electron orbits) with mathematical operators - tables of transition values - connecting discrete quantum states rather than points in space.
Recall that a classical “electron orbit” can be mathematically described by a Fourier series, with each harmonic having a particular frequency and amplitude. The intensity of a spectral line is related to the square of the amplitude of its corresponding Fourier component. When quantities represented by Fourier series are multiplied together, the frequencies combined to produce sum and difference frequencies consistent with the Ritz combination principle. All contributions with the same resulting frequency then had to be added together to yield the new Fourier series.
Bohr’s correspondence principle then demanded that the quantum description should reproduce the classical description in the limit of high quantum numbers, but Heisenberg had radically reinterpreted what the Fourier components represented. Instead of components of an unobservable electron orbit, they were quantities associated with transitions between stationary states – the table entry X(nm), for example, represented the amplitude associated with a transition between states m and n. He therefore needed to determine how the classical algebra of Fourier components could be reproduced using these new transition quantities. In particular, when two such quantities were multiplied, he had to determine which combinations contributed to a particular transition frequency.
The Ritz combination principle inherited from the classical picture dictated that the energy difference between, for example, state 1 and state 3 was the same as the sum of that between states 1 and 2, and that between states 2 and 3. More generally, E(nm) = E(nk) + E(km) where ‘k’ is an intermediate state. Thus when Heisenberg multiplied his Fourier-like transition quantities, every intermediate state ‘k’ supplied a product with the correct frequency to contribute to the ‘nm’ component. All such contributions then had to be added together, giving him a new algebraic rule for multiplying his tables.
The Fourier algebra dictates how transition amplitudes must be combined in order to yield the observed spectral line intensities. Heisenberg wasn't trying to find a way to find the square of a table just because intensities are squares of amplitudes – he was trying to preserve the classical algebra of multiplying Fourier-expanded physical quantities after replacing their Fourier components with his transition amplitudes. When He wrote up his work in a paper and presented it to his supervisor Max Born, Born and his student Pascual Jordan recognized that Heisenberg had independently re-invented an existing branch of mathematics called matrix algebra. Moreover, Born and Jordan also realized that this new mathematical framework had profound implications that went beyond spectroscopy.