On July 27th, after over a year of assessing different methods of improving transparency, the ABA Section of Legal Education announced important changes to its plan for collecting employment data. In a memorandum sent to all law school deans and career services officers, Bucky Askew (Consultant on Legal Education) and Dean Art Gaudio (Chair of the Questionnaire Committee) revealed that the Section would begin collecting graduate-level employment data from the law schools, as opposed to merely collecting data in the aggregate (such as the percent of a class employed in a job or the percent who passed the bar). The memorandum is attached to this post below. Once finalized, the Section of Legal Education would become responsible for collecting hundreds of thousands of data points each year, a task which has historically been undertaken by NALP.
Currently, the Section of Legal Education does not collect granular employment data from law schools. Instead, schools must only report very basic information about the entire graduating class on the annual questionnaire (read more here), a practice which has permitted widespread misunderstandings about the nature of the entry-level hiring market. NALP, on the other hand, annually administers its own detailed survey to gather data about individual graduates.
Despite the voluntary nature of reporting these data to NALP, an overwhelming percentage of ABA-approved law schools (192 out of 199) take the time to do so. From this sizable dataset, NALP cleanses the data for discrepancies, generates private reports for each school to assess its own performance, and creates general reports about the state of entry-level legal hiring. Schools then have the option of using the private NALP-generated reports to respond to the U.S. News survey and ABA Annual Questionnaire. Schools can (but don’t) choose to release these reports to members of the public, specifically to prospective law students. NALP does not release any school-specific information due to agreements it has with its member schools.
As we reported in June, the Section of Legal Education has already made important strides, in its adoption of a new policy that will increase the amount of information available about each school’s entire graduating class. We said:
[T]his solution will provide prospectives a more thorough and accurate window into the placement opportunities at various law schools. However, we do not think this solution goes far enough with consideration to our goal of helping prospective law students find the schools that best meet their individual career objectives. The adopted solution does not provide enough graduate-level detail for those seeking to make a meaningful connection between the post-graduate outcomes for a given school’s graduates and the job characteristics, including salary, required credentials, and location. It is still too difficult to connect these dots.
While the Section of Legal Education has not decided to share the graduate-level data with consumers, a finalized policy would indicate that the Section understands the importance of the underlying data, not only for understanding the aggregate information schools report, but for enabling auditing and safeguarding against fraud.
Section officials are aware that LST supports the Section in its decision to collect graduate-level data from the law schools. The Section has failed to adequately regulate law schools on certain issues for too long, and this will contribute to the Section fulfilling its regulatory function. Accreditation is at its core a matter of consumer protection and the rules governing accreditation accordingly need to be sufficiently robust to protect consumers. The Section owes this duty to the profession and to those who wish to enter it. It is not in the profession’s or prospective students’ interest to have anybody misled or ripped off.
In a letter to Mr. Askew and Dean Gaudio, NALP’s executive director, Jim Leipold, expressed in no uncertain terms his anger over the Section of Legal Education decision to begin collecting graduate-level employment data in this manner.
We object to this action on several grounds, including the fact it will actually lead to LESS transparency and information about the entry-level legal employment market and not more, and the fact that it is an action that is contrary to all of the public conversations about this issue that have taken place among the ABA, NALP, the law schools, and the public over the last year and a half.
. . .
One of the chief harms caused by this action is that it will require a dual reporting burden by the law schools, who now will be asked to report individual student record level employment data to both the ABA and NALP.
. . .
Worst, we fear, is that if schools are required to separately report employment outcomes to the ABA, there is a great risk that many of them will no longer report their data to NALP. This will inevitably lead to the reduction in the amount of information we have about the entry-level legal employment process, and will have the long-term effect of producing less transparency about the legal job market and not more.
These are important concerns. NALP has provided useful information about the entry-level legal market for 37 years, and there is certainly risk that some schools will stop reporting to NALP due to the dual burden. Such big-picture analyses of the health of the legal industry are useful for schools and employers trying to gauge larger trends in hiring shifts, but they cannot be expected to replace the work of an accrediting agency. If the Section of Legal Education is finally deciding to fulfill its accreditation responsibilities fully, this decision should be given a certain level of deference. In other words, if only one group receives the underlying data, it should be the accrediting body and not the third-party relying on voluntary reporting (which itself is enhanced by privacy agreements that make the data inaccessible to those who need it).
However, this is a false dichotomy. There is no need for the ABA Section of Legal Education and NALP to clash; the two can co-exist seamlessly. NALP collects an enormous number of data each year and not only is the process well thought out, but the definitions are useful, coherent, and authoritative. NALP also already collects almost all of the data the Section can reasonably desire to collect itself. In the interim minor differences will exist, such as conflicting definitions of what jobs qualify as short-term employment, but there is little reason to believe that NALP would not be willing to negotiate the terms if the two groups reached a mutually beneficial understanding. However, this requires reopening the discussion and mending the relationship with NALP quickly so that the annual questionnaire may go out at the end of this month.
Recognizing the important and historically complimentary roles of both the Section of Legal Education and NALP, we believe that a compromise in the collection of employment data is both achievable and desirable. Our suggestions for reaching such a compromise utilize the following premises:
- That the Section of Legal Education actually does wish to collect employment data at graduate-level detail.
- That it is important for the Section of Legal Education to fulfill its accreditation obligations, which encompass the collection of employment data at graduate-level detail, so as to limit fraud and enable auditing where such auditing is shown to be necessary.
- That NALP already collects these data and more.
- That at least some law schools will not participate in NALP’s survey under the Section’s proposed changes, because they believe doing so would be too costly.
- That if fewer law schools participate in NALP’s voluntary survey, it will damage NALP’s ability to provide systemic employment information to schools, the legal profession, and prospective law students.
- That, if in the end only one of the Section of Legal Education and NALP can collect employment data, it should be the Section of Legal Education.
- That both the Section of Legal Education and the ABA should respect and value NALP’s longstanding service to the profession and engage in dialogue with NALP’s leadership.
- That, if possible, NALP’s function should be preserved.
- That the Section of Legal Education can fulfill its accreditation responsibilities by using the questions and definitions NALP has fashioned over the years, and does not need to reinvent the wheel.
It’s important to remember what the real fight has been about when discussing law school transparency: the optimal level of information. Schools already collect enough data to more than adequately inform prospective law students. Yet, these data remain private and inaccessible to those who genuinely need quality information. As such, the success of any reforms hinges on the quality of information that follows after schools report data. But this is not the controversy before us today. This is a clash over who can collect and access the underlying data.
As we said above, this clash is unnecessary. The Section of Legal Education and NALP need to work together, not deride each other in the press and behind closed doors. NALP should communicate a willingness to cede the final say on post-graduation outcome surveys, and in exchange continue to gain access to the data.
The simplest (and also cheapest) way to achieve this is for NALP and Section of Legal Education to use the same survey. Under this model, NALP would use the Section of Legal Education’s survey that happens to be based on NALP’s survey. Each year, NALP and the Section can discuss changes, but the Section would have the final say. This does not constitute outsourcing a regulatory function to NALP, something the Council of the Section of Legal Education legitimately fears doing, but it does recognize and utilize NALP’s great work over the past 37 years. NALP’s role would diminish only as far as the Section does not defer to its institutional expertise in making changes to the survey.
The Section has the power to put NALP out of the employment statistics business, and it should not wield this power irresponsibly. But it should also not forget that it has a responsibility to the profession and to those who wish to enter into it, and that some of this responsibility can be shouldered by NALP without outsourcing its regulatory function. Adopting NALP’s survey and inviting NALP to help change it in the future is the right thing to do given the obvious pressure to better regulate law schools. There would still be details to work out. For instance, the Section would need to invest significant resources into technology (especially to ensure that NALP and the Section do not end up with different data) and staff. Similarly, NALP would need to share its wisdom and processes for cleansing the employment data.
This is an important problem that needs to be solved immediately. NALP contributes a great deal to the legal profession, and the Section wants to enhance its own contribution. Through this particular compromise, the two groups can maximize contribution and continue a long-standing relationship.