One of the biggest unsolved problems in the music business today is the lack of authoritative mapping between music recordings and the compositions they embody. When music people talk about the industry’s “database problem,” this is a big part of it. In the past, the industry has managed to work around it in various ad hoc ways. But as interactive streaming services like Spotify, Apple Music, and Google Play have risen in importance, it has been exposed as a gaping hole.

Last week a proposed settlement was announced in a potentially huge putative class action suit against Spotify. The settlement agreement shows how big this problem has become, how the industry’s workarounds are failing, and the growing sense of urgency to solve it.

The problem stems from the fact that each track of recorded music you buy, download, hear on the radio, or stream has (at least) two copyrights: one in the musical composition being recorded (which you can think of as sheet music), the other in the sound recording of that composition. (With mashups and remixes, a sound recording can embody more than one composition.) Although copyrights have existed in musical compositions for a long time, copyrights in sound recordings are more recent — for the simple reason that sound recording technology is itself more recent. Hence two separate copyrights, and two separate sets of business practices and laws around each of them.

One big difference between the recorded music and music publishing industries is that record labels have long interfaced with consumers through retailers, while music publishers haven’t.  This has forced labels to keep track of things like how many of which records sold at retail stores and whom to pay royalties (and how much) for those sales. In contrast, music publishing is really a B-to-B business, one that hasn’t required the same transaction-level precision of data. Instead, music publishers — and their associates, such as collecting societies — have sometimes relied on various types of deals based on revenue percentages, sample data, blanket licenses, and other mechanisms that don’t necessarily rely on precise data about music usage, as well as data that record labels feed them from retail sales.

As a result, today we have a music industry with one part (recorded music) that has pretty good data and is able to handle large volumes of transactions fairly well, and another (music publishing) that doesn’t and isn’t.

This didn’t matter so much until interactive streaming became important over the past few years. Interactive streaming has exposed and exacerbated the recording-composition linking problem because it requires orders of magnitude more transactions with precise data to be processed than ever before. In particular, songwriters and music publishers now need to be paid with precision on billions of transactions per year.

Songwriters and music publishers get two types of royalties from streaming services: performance and mechanical (reproduction). Performance rights organizations (PROs) like ASCAP and BMI have some flexibility in royalty processing through blanket licensing agreements with streaming services. But there are no blanket licenses for mechanicals.

Interactive streaming services are expected to offer just about every song ever recorded, but they only have information about underlying compositions for a subset of them. They get music from record labels, which often don’t keep track of composition information, yet they are supposed to pay mechanicals on those compositions.

The music services have tried to solve this problem by hiring rights management services that painstakingly build and maintain their own sets of data — companies like Harry Fox Agency (HFA), Music Reports Inc. (MRI), and MediaNet — to find the compositions and pay royalties on them. (Google acquired a similar company, RightsFlow, to do this for YouTube and Google Play.) Yet these proprietary databases can’t completely keep up with the major interactive streaming services’ vast and fast-growing catalogs.

Meanwhile, interactive streaming services can legally play any song that a record label sends them without obtaining licenses directly from music publishers. Copyright law gives streaming services compulsory mechanical licenses to the compositions underlying sound recordings as long as they file “Notices of Intention” (NOIs), even if they don’t know the composer or publisher; they are expected to put the money aside in order to pay them later.

This system ran along for a while. Third-party rights managers ensured that the big guys got paid, while some long-tail songwriters and publishers not affiliated with the majors didn’t. This wasn’t a big deal… until interactive streaming grew to become a major source of revenue in the music industry. Then the indies started to notice, and they didn’t like what they saw.

A few of them started to sue, and Spotify was the main target. The National Music Publishers Association (NMPA), the largest trade association for U.S. music publishers, negotiated a settlement with Spotify a year ago, reportedly worth $30 million; but the NMPA settlement is limited to NMPA members and then only on an opt-in basis. Individual songwriters filed lawsuits with a view to launching a class action. Plaintiffs included indie-rock pioneer and artists’ rights advocate David Lowery, singer-songwriter Melissa Ferrick, and the estate of legendary bassist Jaco Pastorius. If the plaintiffs had succeeded in getting a judge to certify a class action and then prevailed, it could have had a ripple effect to other services besides Spotify and brought the entire U.S. interactive streaming music scene to its knees.

The lawsuit with Ferrick as lead plaintiff, filed in January of last year, contained two pertinent allegations: that Spotify hired HFA to match recordings to compositions, but HFA didn’t do a good job of it; and that Spotify failed to file many required NOIs. This is the case in which a settlement agreement was made public last week. The agreement is subject to approval by the judge in the case, the timing of which is unclear.

As is typical in these cases, Spotify isn’t admitting any wrongdoing. Yet the settlement goes further than simply paying the plaintiffs to go away. It attempts to solve the problem of unpaid mechanicals in two ways.

First, it requires Spotify to make its track and stream data available through a web interface so that rights holders can search it and claim royalties. Spotify will allow songwriters, publishers, and their representatives to log into it to find their compositions, determine how many streams they have gotten, and claim royalties that will be paid out from a $43 million settlement fund.

Rights holders will be able to use the services of a “Settlement Claim Facilitator,” a rights management provider that the plaintiffs’ counsel will select to find recordings in the Spotify catalog that match their compositions. MRI — which performs this type of service for Sirius XM, Pandora, and others — is already in consideration for this role. Although the Settlement Claim Facilitator will be forbidden from using any data from the Spotify track database for any other purpose, it may be able to use data it gets from claimants to add to its database of matches between recordings and compositions — and given the sheer size of Spotify’s catalog, thereby gain an advantage over its competitors.

Second, the settlement calls for Spotify to invite big industry players — major publishers, PROs, competing streaming services, and the RIAA — to form a “Copyright Data Sharing Committee” to confidentially discuss sharing catalog data with a view to making mechanical licensing easier and more accurate. If at least four such entities agree to show up, the committee has to exist and meet regularly for at least two years, though it’s under no legal obligation to actually accomplish anything.

Yet one concrete thing this committee could do is take advantage of the work of the Open Music Initiative (OMI) on interoperability of information on music and music rights holders. (I’m an OMI member.) The OMI is close to publishing a set of requirements and uses for “minimum viable interoperability” (MVI), which will then lead to a set of APIs for enabling identification of musical compositions, sound recordings, composers, and recording artists for purposes of communicating that information among systems… like those of streaming music services, labels, PROs, and music publishers. If the Copyright Data Sharing Committee actually forms and agrees to use the OMI’s MVI APIs, it will have a way of sharing catalog data that ensures that the data is sufficient yet each entity only has to expose the minimum amount of data necessary.

However, interoperability of data is one thing; creating complete and correct data and making it available are entirely different matters, and they’re anything but trivial. One of the points of contention in the disputes against Spotify has been whose responsibility it should be to determine composition mappings for recordings. The settlement agreement requires Spotify to pay for the building of the web interface to its data, but otherwise, the effort and expense of searching it, finding matching compositions, and making claims for royalties all go to the rights holders. This particular division of responsibility may be mandated by a legal document, but it’s an example of the type of cooperation that’s going to be necessary in the future if the recording-composition mapping problem is ever going to be solved.