What is the significance of the S-matrix in physics?

The S-matrix is an asymptotic operator which describes how particles going into a scattering event transform into particles going out. The S-matrix can be calculated from a Hamiltonian description, but the nice thing about it is that it does not require a Hamiltonian or Lagrangian description of the intermediate details of the scattering at all, it can be built up without regard to the local space-time structure. Because of this, you can use it to construct theories which are insensitive to a breakdown of naive space-time structure. You don't need any knowledge of the local structure of space and time to talk about incoming and outgoing particles, since these are defined at far away locations and far away times, so you know they can be described in the ordinary way, using plane waves.

From the S-matrix idea, you can reconstruct physics, but you need some assumptions. Feynman started with the idea of an electron and a photon, and classical electrodynamics in the classical limit, and found the Feynman rules for QED. Schwinger and Dyson found the same rules from the Hamiltonian description of QED, and it required a renormalization procedure to make sense of the diagrams in both pictures. So Feynman decided S-matrix was equivalent to field theory, and stuck with field theory for the rest of his career.

But others pursued a pure S-matrix theory. Chew and Mandelstam, working with consistency conditions, decided that there was enough information in the S-matrix to reconstruct all of physics. People worked hard throughout the 60s to show how this program would work, and a lot of people accepted this, but a lot of people also stuck to field theory too. At the time, the focus was the strong interaction.

The S-matrix description of Pions and Nucleons developed by Chew in the early 1960s transmuted into the effective field theory of Nambu and Weinberg in the late 1960s. Weinberg became convinced that the only solution to the S-matrix consistency conditions was a form of field theory, and he was sort of right, under the assumption of finitely many fundamental particles.

But Tullio Regge showed that particles can come in families, and Chew and Mandelstam persisted in looking for a theory of exchange of Regge trajectories. Vladimir Gribov described these Regge proceses with a calculus of diagrams, but this calculus ultimately had an interpretation in terms of a two-dimensional light-cone picture developed by Feynman, Gribov, and later followed up by Kenneth Wilson and nowadays is developed further by Sarada Rajeev. It still wasn't a new theory.

But in 1968, Veneziano found a formula for an S-matrix approximation (a first-order scattering amplitude) that was clearly completely different from field theory. This was the foundation of string theory, it developed into string theory over the following decades.

In the mid 1990s, the S-matrix picture was understood more completely as the form of holographic principle appropriate to asymptotically flat space time. The statement that "everything is in the S-matrix" is then more properly reinterpreted as the statement that the local physics is reconstructed from dynamics on holographic boundaries at the edge of the universe. This became accepted when it was demonstrated to work in AdS/CFT models, and now all this old stuff is water under the bridge. But betwee 1960 and 1974, there were two camps in physics who hated each other and did not hire each other, or read each other, the S-matrix people and the field theory people.

Both sides made spectacular progress, but the S-matrix folks made more progress and got beat up more for it, so I prefer to laud them more.