All News in ABA Watch

Paul Pless

The Chicago Tribune reported over the weekend that an admissions dean at University of Illinois College of Law has been put on leave pending the outcome of an investigation into the fraudulent reporting of admissions data on the Illinois website. While the data does not appear to have been submitted to the ABA Section of Legal Education, this would still violate Standard 509 because it covers all basic consumer information, not just information submitted to the Section.

Above the Law reports that the investigation will be conducted by Theodore Chung of Jones Day. NonTradLaw reports that the dean is Paul Pless. Confirming this suspicion, Mr. Pless’s profile has disappeared from the Illinois website.

According to Charles Cooper from NonTradLaw:

We need law schools, we need new lawyers, but we don’t need vast numbers of uninformed, indebted, and unhappy law grads. And inflated stats, to some extent, draw in applicants who have no place in law school. This has to stop, at this point, it’s got to stop sooner rather than later.

He is, of course, correct. We expect this to reignite the discussion about auditing all consumer data. While the task will be very difficult and costly for employment data, admissions data would be extremely cheap to audit if the Law School Admissions Council were to cooperate. All ABA-approved law schools are members. Representatives of LSAC have said that LSAC is not interested in auditing admissions data, despite presently having the capabilities to do so.

The Council of the Section of Legal Education announced today that it would move forward in collecting graduate-level data from law schools. As we reported yesterday, the Section and NALP will collaborate as to limit the negative effects of this policy on NALP’s annual studies of the entry-level hiring market.

As we discussed Wednesday, the relationship between the ABA Section of Legal Education and NALP has been rocky over the last week. Members from both organizations made statements to the press and bystanders asked whether this would be the end of NALP. Concerned with that possibility, we sent a letter to the Section and to NALP urging the two to compromise.

Now, the ABA Journal reports that NALP and the Section of Legal Education have decided to cooperate:

On Friday, both groups spoke of their long history of collaboration on data collection and production, particularly on employment and placement figures, according to Hulett “Bucky” Askew, the ABA’s consultant on legal education. Both NALP and the council spoke of the desire to continue to work together moving forward.

Although Askew did not confirm whether an agreement had been met on the methodology the section would use to collect data directly from the schools, a prior point of contention for NALP, he did say that the two groups will continue to meet and discuss ways to address the needs of all parties.

This is reassuring news for the Section of Legal Education, NALP, prospective law students, and the legal profession as a whole. We will monitor the situation closely and report on the compromise when more details are known.

• LST Letter to Section of Legal Education and NALP

On July 27th, after over a year of assessing different methods of improving transparency, the ABA Section of Legal Education announced important changes to its plan for collecting employment data. In a memorandum sent to all law school deans and career services officers, Bucky Askew (Consultant on Legal Education) and Dean Art Gaudio (Chair of the Questionnaire Committee) revealed that the Section would begin collecting graduate-level employment data from the law schools, as opposed to merely collecting data in the aggregate (such as the percent of a class employed in a job or the percent who passed the bar). The memorandum is attached to this post below. Once finalized, the Section of Legal Education would become responsible for collecting hundreds of thousands of data points each year, a task which has historically been undertaken by NALP.

Currently, the Section of Legal Education does not collect granular employment data from law schools. Instead, schools must only report very basic information about the entire graduating class on the annual questionnaire (read more here), a practice which has permitted widespread misunderstandings about the nature of the entry-level hiring market. NALP, on the other hand, annually administers its own detailed survey to gather data about individual graduates.

Despite the voluntary nature of reporting these data to NALP, an overwhelming percentage of ABA-approved law schools (192 out of 199) take the time to do so. From this sizable dataset, NALP cleanses the data for discrepancies, generates private reports for each school to assess its own performance, and creates general reports about the state of entry-level legal hiring. Schools then have the option of using the private NALP-generated reports to respond to the U.S. News survey and ABA Annual Questionnaire. Schools can (but don’t) choose to release these reports to members of the public, specifically to prospective law students. NALP does not release any school-specific information due to agreements it has with its member schools.

As we reported in June, the Section of Legal Education has already made important strides, in its adoption of a new policy that will increase the amount of information available about each school’s entire graduating class. We said:

[T]his solution will provide prospectives a more thorough and accurate window into the placement opportunities at various law schools. However, we do not think this solution goes far enough with consideration to our goal of helping prospective law students find the schools that best meet their individual career objectives. The adopted solution does not provide enough graduate-level detail for those seeking to make a meaningful connection between the post-graduate outcomes for a given school’s graduates and the job characteristics, including salary, required credentials, and location. It is still too difficult to connect these dots.

While the Section of Legal Education has not decided to share the graduate-level data with consumers, a finalized policy would indicate that the Section understands the importance of the underlying data, not only for understanding the aggregate information schools report, but for enabling auditing and safeguarding against fraud.

Section officials are aware that LST supports the Section in its decision to collect graduate-level data from the law schools. The Section has failed to adequately regulate law schools on certain issues for too long, and this will contribute to the Section fulfilling its regulatory function. Accreditation is at its core a matter of consumer protection and the rules governing accreditation accordingly need to be sufficiently robust to protect consumers. The Section owes this duty to the profession and to those who wish to enter it. It is not in the profession’s or prospective students’ interest to have anybody misled or ripped off.

NALP’s Concerns

In a letter to Mr. Askew and Dean Gaudio, NALP’s executive director, Jim Leipold, expressed in no uncertain terms his anger over the Section of Legal Education decision to begin collecting graduate-level employment data in this manner.

We object to this action on several grounds, including the fact it will actually lead to LESS transparency and information about the entry-level legal employment market and not more, and the fact that it is an action that is contrary to all of the public conversations about this issue that have taken place among the ABA, NALP, the law schools, and the public over the last year and a half.

. . .

One of the chief harms caused by this action is that it will require a dual reporting burden by the law schools, who now will be asked to report individual student record level employment data to both the ABA and NALP.

. . .

Worst, we fear, is that if schools are required to separately report employment outcomes to the ABA, there is a great risk that many of them will no longer report their data to NALP. This will inevitably lead to the reduction in the amount of information we have about the entry-level legal employment process, and will have the long-term effect of producing less transparency about the legal job market and not more.

These are important concerns. NALP has provided useful information about the entry-level legal market for 37 years, and there is certainly risk that some schools will stop reporting to NALP due to the dual burden. Such big-picture analyses of the health of the legal industry are useful for schools and employers trying to gauge larger trends in hiring shifts, but they cannot be expected to replace the work of an accrediting agency. If the Section of Legal Education is finally deciding to fulfill its accreditation responsibilities fully, this decision should be given a certain level of deference. In other words, if only one group receives the underlying data, it should be the accrediting body and not the third-party relying on voluntary reporting (which itself is enhanced by privacy agreements that make the data inaccessible to those who need it).

However, this is a false dichotomy. There is no need for the ABA Section of Legal Education and NALP to clash; the two can co-exist seamlessly. NALP collects an enormous number of data each year and not only is the process well thought out, but the definitions are useful, coherent, and authoritative. NALP also already collects almost all of the data the Section can reasonably desire to collect itself. In the interim minor differences will exist, such as conflicting definitions of what jobs qualify as short-term employment, but there is little reason to believe that NALP would not be willing to negotiate the terms if the two groups reached a mutually beneficial understanding. However, this requires reopening the discussion and mending the relationship with NALP quickly so that the annual questionnaire may go out at the end of this month.

LST’s Suggestion

Recognizing the important and historically complimentary roles of both the Section of Legal Education and NALP, we believe that a compromise in the collection of employment data is both achievable and desirable. Our suggestions for reaching such a compromise utilize the following premises:
- That the Section of Legal Education actually does wish to collect employment data at graduate-level detail.
- That it is important for the Section of Legal Education to fulfill its accreditation obligations, which encompass the collection of employment data at graduate-level detail, so as to limit fraud and enable auditing where such auditing is shown to be necessary.
- That NALP already collects these data and more.
- That at least some law schools will not participate in NALP’s survey under the Section’s proposed changes, because they believe doing so would be too costly.
- That if fewer law schools participate in NALP’s voluntary survey, it will damage NALP’s ability to provide systemic employment information to schools, the legal profession, and prospective law students.
- That, if in the end only one of the Section of Legal Education and NALP can collect employment data, it should be the Section of Legal Education.
- That both the Section of Legal Education and the ABA should respect and value NALP’s longstanding service to the profession and engage in dialogue with NALP’s leadership.
- That, if possible, NALP’s function should be preserved.
- That the Section of Legal Education can fulfill its accreditation responsibilities by using the questions and definitions NALP has fashioned over the years, and does not need to reinvent the wheel.

It’s important to remember what the real fight has been about when discussing law school transparency: the optimal level of information. Schools already collect enough data to more than adequately inform prospective law students. Yet, these data remain private and inaccessible to those who genuinely need quality information. As such, the success of any reforms hinges on the quality of information that follows after schools report data. But this is not the controversy before us today. This is a clash over who can collect and access the underlying data.

As we said above, this clash is unnecessary. The Section of Legal Education and NALP need to work together, not deride each other in the press and behind closed doors. NALP should communicate a willingness to cede the final say on post-graduation outcome surveys, and in exchange continue to gain access to the data.

The simplest (and also cheapest) way to achieve this is for NALP and Section of Legal Education to use the same survey. Under this model, NALP would use the Section of Legal Education’s survey that happens to be based on NALP’s survey. Each year, NALP and the Section can discuss changes, but the Section would have the final say. This does not constitute outsourcing a regulatory function to NALP, something the Council of the Section of Legal Education legitimately fears doing, but it does recognize and utilize NALP’s great work over the past 37 years. NALP’s role would diminish only as far as the Section does not defer to its institutional expertise in making changes to the survey.

The Section has the power to put NALP out of the employment statistics business, and it should not wield this power irresponsibly. But it should also not forget that it has a responsibility to the profession and to those who wish to enter into it, and that some of this responsibility can be shouldered by NALP without outsourcing its regulatory function. Adopting NALP’s survey and inviting NALP to help change it in the future is the right thing to do given the obvious pressure to better regulate law schools. There would still be details to work out. For instance, the Section would need to invest significant resources into technology (especially to ensure that NALP and the Section do not end up with different data) and staff. Similarly, NALP would need to share its wisdom and processes for cleansing the employment data.

This is an important problem that needs to be solved immediately. NALP contributes a great deal to the legal profession, and the Section wants to enhance its own contribution. Through this particular compromise, the two groups can maximize contribution and continue a long-standing relationship.

• Memorandum to Law Schools on New Policy

Senator Chuck Grassley, the Ranking Member of the Senate Committee on the Judiciary, recently sent a letter to Stephen Zack, President of the American Bar Association. The letter focuses on a recent accreditation review of the ABA Section of Legal Education conducted by the National Advisory Committee on Institutional Quality and Integrity (NACIQI), which found numerous problems with the Section in its Department of Education-delegated regulatory capacity. While many of the problems are technical and easy to correct, NACIQI members were frustrated with the level of noncompliance and a few were vocal with their concerns.

Senator Grassley’s Letter

Senator Grassley’s letter contains a list of questions regarding whether and how the ABA regulates certain aspects of J.D. programs, intimating that the ABA needs “stronger oversight controls.” The Senator inquired into the collection and disclosure of scholarship retention rates (which recently gained public awareness), the collection and disclosure of loan default rates, ABA programs dedicated to educating consumers about debt repayment, and disciplinary proceedings against individual law schools.

But perhaps most interesting is the line of questioning concerning whether the ABA “track[s] the professional background of its committee membership” for “committees related to the accreditation of law schools.” Qualifying committees include not only the Accreditation Committee but also the Standards Review Committee and the Questionnaire Committee, both of whom are actively involved in redesigning how law schools collect and report employment data about graduates. It also includes the supervisory Council of the Section of Legal Education, which must vote to approve or reject committee proposals before they become enforceable.

These committees and the Council consist primarily of law school academics, deans, former deans, university presidents, and legal counsel who have been employed or are currently employed by law schools or universities. Some of these designations skirt conflict of interest rules even though they still indicate involvement in the law school model.

The ABA submitted its responses to Senator soon afterwards, one from Mr. Zack and another from the ABA Section of Legal Education. As both the letter and the responses indicate, we are seeing the reemergence of an old discussion about the professional backgrounds of ABA committee members and their role in the adaptation and development of legal education. This is a decades-long discussion about the nature of legal education and the arguably protectionist image of its accrediting body. With two U.S. senators now turning the public’s eye on these issues, Mr. Zack and the ABA cannot be comfortable with the level of congressional scrutiny regarding the Section of Legal Education and its various committees.

U.S. v. American Bar Association, 1995

As The Legal Dollar points out, committee membership rules were established following a 1995 settlement between the ABA Section of Legal Education and the Department of Justice. The rules aim to limit the number of committee members who can be directly employed by a law school at the time they serve on the committee. The Legal Dollar offers some interesting commentary as to why the Section of Legal Education has not complied with the spirit of the settlement.

We won’t repeat that discussion here. Rather, we call attention to two passages from the DoJ’s Competitive Impact Statement that we believe add context to the Section’s response to Senator Grassley:

The Complaint also alleges that the ABA allowed its law school accreditation process to be captured by those with a direct interest in its outcome. Consequently, rather than setting minimum standards for law school quality and thus providing valuable information to consumers, the legitimate purposes of accreditation, the ABA at times acted as a guild that protected the interests of professional law school personnel.

. . .

Legal educators, including current and former law school deans, faculty, and librarians, control and dominate the ABA’s law school accreditation process. Approximately 90% of the Section of Legal Education’s members are legal educators.… All current members of the Standards Review Committee and a majority of the current members of the Accreditation Committee are legal educators.

The Department of Justice thus drew two important distinctions regarding the accreditation of law schools in making its complaint. The first distinction is between the goals of a legitimate accrediting agency and the goals of a captured one: providing consumers with valuable information about the quality of a law school (the legitimate goal in this instance), vs. protecting the interests of law school faculty and staff. The second distinction is between the types of committee member employment that lend themselves to the existence of a captured agency and the types that do not. It’s important to note that the “legal educators” whom the DoJ accused of capturing the Section of Legal Education back in 1995 included former law school deans and faculty. The final consent decree also included law school staff but excluded former employees and university employees from the set of “captured” employment. As we argue below, these distinctions cause the Section’s committees to possess an appearance of impropriety, although whether actual impropriety exists is up for debate.

Prior to the consent decree, the Department of Justice noted that a majority of the Accreditation Committee were current or former legal educators. Seventeen years later, we still have a majority of current and former legal educators running the show. The 2010-2011 Accreditation Committee is comprised of 19 members. In the Section’s response to Senator Grassley, the Section breaks down committee membership as consisting of nine academics (law school professors or deans), five practicing lawyers, three public members, one judge, and one university president. Under the strict terms of the settlement decree, this does not violate the rule against having a majority of academics serving on any particular committee.

In reality, more than half of the members labeled as something other than “academics” have a direct interest in the present law school education model. For starters, four members have been associated with university systems that contain affiliated law schools (two as general counsels, one as vice chancellor, and one as a university president). Universities play an important role in law school finances, driving up the costs of attendance by depending on law school tuition dollars to fund other programs within the system. Additionally, two of the other non-academics are former law school deans. Perhaps former deans aren’t collecting a paycheck from one of the schools they are now regulating, but one would be naive to assume these accomplished leaders within the academy have severed all ties and allegiances. When nearly 80% of a regulatory committee consists of people who built their careers within a law school or an affiliated university, it is no surprise to see people questioning the committee’s independence.

While Senator Grassley has not yet explained why he is so interested in examining the professional background of committee members, it’s reasonable to assume his concern deals with agency capture by “those with a direct interest” in the accreditation process. His entrance into the debate has put an interesting twist on the breaking trust relationship between law schools and their students, their graduates, and the profession, something we’ve pointed out before.

The Breaking Trust Relationship

We do not dispute that some faculty members involved in accreditation are dedicated proponents of reform. LST has acknowledged the Section of Legal Education’s important prioritization of law school transparency over the last year, and we are supportive of the individuals who have dedicated so much time attempting to resolve some of the most pressing issues. But as the public debate about education continues to unfold, law schools cannot and should not be viewed separately from their role as the gateway into the legal profession.

In this role, both the schools and the ABA Section of Legal Education are failing in their responsibilities. Schools have a duty to adequately inform potential consumers about the value of a degree program. And the Section of Legal Education has a duty to reform legal education when the schools it accredits do not meet the needs of the profession. As the Department of Justice made clear in its antitrust suit nearly two decades ago, the purpose of an accreditation committee is to protect consumers by ensuring a level of quality. Necessary to this protection is determining how to measure the quality of a program, which is intrinsically linked to the outcomes of its graduates in the entry-level job market (for reasons we have discussed before). Those involved in law school accreditation must be more diverse in their backgrounds, particularly as the academy’s constituents do not have more than a nominal amount of experience in legal practice.

Next Up: Improving Legal Education

Senator Grassley may call for the legal profession to play a different role in regulating law schools directly. Further investigation could lead to structural reforms in how the Section of Legal Education operates. This prompts an interesting question: what’s the appropriate mix of professional backgrounds for people serving on these committees?

For starters, more consumer representation is critical. The consumer group includes not only prospective and current students, but also employers who hire or would like to hire recent graduates. The Section of Legal Education currently allows for only one member of the Council to be a Law Student Division Member. No other student representatives serve on any of the other committees. How can one consumer representative be enough to ensure fair play, given that the majority of the accusations levied against law schools deal with how they are misleading and defrauding students? A better mix might therefore mandate greater student (or perhaps recent graduate) membership to protect the rights and needs of consumers.

Second, to the extent that law school employees continue to serve minority roles on these committees, we should consider drawing a distinction between classroom-focused academics and the people who develop and provide practical skills and job placement assistance. This latter group might include career services officers, bridge-to-practice administrators, adjuncts who spend the majority of their careers in actual practice, and clinical professors. A regulatory agency charged with overseeing institutions should have experience in all aspects of how those institutions work, and traditional classroom instruction and scholarship are only two aspects of a legal education. Further, these aspects are increasingly being called into question. Law schools offer a host of professional services designed to prepare students for actual practice or assist them in finding a job, for which a measure of quality necessarily includes providing the consumer with information about results. Even where faculty do play a role in developing these services (most often while serving as dean), they do not generally know how the results of those services are advertised to prospective law students.

Finally, the inclusion of more practitioners with relevant experience would inject new leadership into the Section of Legal Education. To accomplish this goal, the ABA should revisit whether its ethical and professional leadership requires a shift in how it oversees legal education. ABA members play an active role in many aspects of the profession, in ways that could be directed to the benefit of current and prospective students. Attorneys who understand the legal hiring market for new graduates can (and at some schools already do) offer guidance in fixing the educational model to be more apprenticeship-based. These fixes should be taking place at the accrediting level, not just within individual schools. Regulators experienced in handling consumer protection claims are well-situated to take a closer look at reviewing admissions brochures and determining whether schools are misleading applicants. Enforcement of the standards will only improve as committee membership includes more attorneys who are familiar with traditional consumer fraud claims. And judges and state bar leaders who enforce professional rules of conduct, particularly rules concerning advertising and ethical communications about a lawyer’s services, would be keen to review law school behavior in the same way they review how attorneys solicit clients.

We believe this last point is timely. A lawyer who makes a false or misleading communication about their services is subject to discipline not only because of the harm they can cause to clients but also for the manner in which their actions are perceived to extend to all lawyers, which reflects poorly on the profession and limits access to justice. Were we to examine law school advertising with the same concern for the damage schools are doing in the eyes of the public, we might see very different results coming out of the enforcement arm of the Section of Legal Education. Judging by the significant number of attorneys who have contacted LST to express their support for (and interest in) improving legal education in the U.S, we think there are many people out there who are both qualified and interested in serving on these committees.

A Call For a New MacCrate Report?

For Mr. Zack’s part, having the ABA take on a greater role in reforming legal education is not a new concept. One of the foremost contributors to legal education reform in the last twenty years is none other than former ABA President Robert MacCrate, who was later instrumental in creating the MacCrate Report and now serves as Senior Counsel at Sullivan & Cromwell. (Mr. MacCrate is being honored for his work at this week’s NYSBA reception in Toronto, scheduled to coincide with the ABA’s annual meeting.)

Is it time for another MacCrate Report, one that again grounds itself in consumer rights and the needs of the profession? Such a report could address many important issues: committee membership within the Section of Legal Education; the perceived lack of enforcement; and the advisability of developing new accreditation rules that prioritize cost reductions and efficiency, with an eye toward enabling law schools to reimagine the educational and professional services they offer. Many a law school dean has argued against rules that increase operating costs and prohibit flexibility in the educational model. Most problematic is the notion that while classroom instruction may be uniform across accredited programs and thus have about the same value, the quality of professional services and the job opportunities for students swing widely without a corresponding change in tuition. As Kimber Russell (formerly of Shilling Me Softly) explained:

The ABA accreditation standards require all law schools to operate, essentially, as “luxury models” despite the fact that students from lower-ranked schools have almost invariably never had the same opportunities afforded to graduates of the vaunted Top 14 schools as ranked by USNWR. What this means is that even the lowest-ranked ABA-accredited school with the very worst reputation will still cost most students the same in tuition as the Ivy League institutions.

The Standards Review Committee is already engaged in “outcome-based reform,” but much more will need to be addressed in the coming months.

In Closing

If U.S. senators are concerned that professional ties are limiting the Section’s ability to regulate law schools nearly two decades after the Department of Justice filed suit, perhaps the ABA and the Section of Legal Education should be worried about what’s on the horizon. We expect that Senator Grassley will respond to the ABA and the Section of Legal Education with continued pressure, and that he and his colleagues will continue to shape the debate on law school transparency.

• Senator Grassley's Letter to the ABA
• Section of Legal Education's Response
• ABA President Steven Zack's Response

The ABA Council on Legal Education and Admissions to the Bar completed an enormous step this morning towards helping prospective law students make informed decisions. The Council, which is the sole accrediting body for U.S. law schools, unanimously approved the Questionnaire Committee’s recommended procedures for the improved collection and sharing of employment data. The recommendation is based on last December’s Questionnaire Committee hearing, at which interested parties, including LST’s executive director, presented on the issue of consumer information transparency.

You can review the now-approved recommendation here (pages 22–28), although the vote added two caveats to the recommendation. Under the new policy, for at least one year, the ABA will work with NALP to leverage NALP’s present collection, cleansing, and distribution practices, subject to the ABA and NALP reaching a contractual agreement concerning confidentiality. Working with NALP this year will allow the ABA to avoid unnecessary costs, divergent information, and confusion.

Several Council members were concerned, however, about using a third party—NALP—as part of the Council’s regulatory oversight, thus desire to reach an appropriate contractual arrangement with NALP. Bryan O’Keefe, the Law Student Divison’s student representative to the Council, clarified that these logistical concerns will not hold up publishing the improved data in the upcoming year.

The Council agreed that these issues will not prevent improving data disclosure in the next questionnaire. In the long-term, the Council wishes to exert direct control over the process, either through hiring its own staff or selecting a third-party vendor through a RFP (request for proposal).

The New Disclosure Policy

Caveat One: The Council’s Executive Committee will work with the Questionnaire Committee to reach an appropriate contractual agreement with NALP.
Caveat Two: The Council will begin work as soon as possible to directly collect the relevant data, either through increased staff or an outside vendor.

Job Data

Schools are required to report:

Employment Status (100% of the class will be accounted for with these categories)
Job Credentials: employed in a job requiring bar passage, in a job for which a JD is preferred, in another professional job, in a non-professional job, or in a job of unknown type.
Non-Employed Status: pursuing a graduate degree; unemployed – not seeking or unemployed – seeking; and status unknown.

Employer Type (100% of the class will be accounted for with these categories)
Law Firms: various sizes based on total attorneys at the firm globally (8 total + an unknown category).
Other Employers: business and industry; government; public interest; judicial clerkships; academia; and employer type unknown.

Employment location:
United States: the three states where the most graduates are employed and number employed in each.
International: the number of graduates employed internationally.

Schools will also report data, where applicable, about whether the jobs are full-time/part-time and long-term/short-term, as well as indicate the number of jobs that are funded by the law school or university.

Salary Data

In addition to the placement data, the ABA/LSAC Official Guide will publish state-specific salary information based on graduates reporting from all law schools. According to Questionnaire Committee, only providing school-specific salary data provides “limited and perhaps confusing information” to applicants.

As the Committee also correctly points out, school-specific data is less representative than the state-specific datasets. Moreover, when only ~50% of graduates report a salary, granular categories like “Employed in a Law Firm of 2-10 Attorneys” are unlikely to have sufficient school-specific data to warrant sharing. (This is a flaw we identified with the 509 Subcommittee’s proposal.)

According to Mr. O’Keefe:

By using aggregate data, students will be able to have a more accurate picture of possible salaries.

When you combine the job data and the salary data, applicants will be able to discern the exact job prospects of an individual school, and using the aggregate salary data, develop a solid idea of what that job prospect makes in any given state. The employment location variable will allow students to assess where those graduates end up working— in essence, you will be able to tell where you are most likely geographically to end up getting that job and making that salary.

We agree that this solution will provide prospectives a more thorough and accurate window into the placement opportunities at various law schools. However, we do not think this solution goes far enough with consideration to our goal of helping prospective law students find the schools that best meet their individual career objectives. The adopted solution does not provide enough graduate-level detail for those seeking to make a meaningful connection between the post-graduate outcomes for a given school’s graduates and the job characteristics, including salary, required credentials, and location. It is still too difficult to connect these dots. Nevertheless, it is a monumental improvement that should be celebrated.

Auditing

The Questionnaire Committee also indicates that it will develop ideas for how to improve the accuracy of the data entered by law school graduates and staff.

[A]s an example of a possible response, the ABA might require that annually or at the time of the sabbatical site visit there be random audits of placement data submitted in the annual surveys. If performed at the time of the site visit, these audits might be “informal” and performed by a member of the site team. We have serious concerns, however, with the ability of a site team member to perform this function in the context of a site visit.

We share these concerns. Site team members do not typically have auditing expertise and usually only collect facts from and about the law school. Mechanically, the site team likely would only examine the data collected by the law school to compare to what the school has reported. This creates an echo chamber, duplicates NALP’s data cleansing (which aims to catch mistakes), and would therefore provide only a marginal return on investment. Villanova-style lying would still be too easy to achieve, although even a single set of third-party eyes might deter schools.

An alternative might be to require schools, on a random basis, to provide a more “formal” audit performed by a CPA firm of their placement survey responses. These random audits could be performed annually or at the time of the site visit. Obviously, this would involve greater expense and we would have to look at ways that the expense could be minimized and distributed among law schools, whether they are audited or not. Finally, there may be other methods of performing such an audit, or an alternative to it; we will consider them also.

As is clear from these suggestions, the idea that all law schools should be subject to yearly auditing is not on the table. It seems unnecessary to us because we do not believe the problem lies with falsified data, but misleading information. An effective alternative, which we’ve already shared with the ABA committees, would be to use the LST Proposal. Through sufficient disaggregation, individual graduates could verify how the school reported their outcomes from behind a veil of anonymity provided by the proposal’s structure. This provides a nearly zero-cost alternative that would deter law schools from fabricating outcomes. Upon an accusation of foul play, the school would tender its source (the survey) and the graduate its evidence of misrepresentation.

Going Forward

The Questionnaire Committee operated with a clear mission:

Our objective in selecting, obtaining, and providing [employment] data are several-fold: (1) to provide correct and complete data (a) to law school applicants to assist them in making decisions on whether to go to law school and, if so, which law school to attend, and (b) to current law students and recent graduates to assist them in making job decisions; and (2) to obtain and provide this information in a way that will require the least amount of additional, unnecessary effort by law schools, particularly in their career services offices.

This echoes LST’s mission and objectives. It’s important that the ABA continues to prioritize law school transparency as legal education continues to change over the next decade.

We look forward to the Questionnaire Committee and Council optimizing the annual questionnaire over the next few years with these objectives in mind because there is still a need for more improvement. Elsewhere within the Section of Legal Education, the Standards Review Committee, operating with similar objectives, continues its work on Standard 509. Together, these approved and prospective changes are a great start.

From the National Law Journal:

Senator Boxer shares our concerns and we appreciate the ongoing dialogue we’re having about the important issues of how law students finance their educations and learn about their employment prospects post-graduation. We’re glad the Senator is “encouraged” and “pleased” by what the American Bar Association and its Council on Legal Education have been doing. – Stephen Zack

Scrutiny of the ABA continues today as Senator Barbara Boxer increases her pressure on the ABA Section of Legal Education’s regulatory failings. Moments ago, she issued her second letter to the ABA on the need for law school transparency (first letter). This letter to ABA President Stephen Zack addresses why Senator Boxer remains concerned despite the ABA’s current efforts, and asks the ABA to explain their plans regarding a few key concerns (full text below).

Senator Boxer would like to see the ABA address:

• the auditing of law school data (noting that current proposals continue to allow self-reporting without auditing procedures);
• better regulation of how prospective students can access information, focusing primarily on how law schools advertise employment outcomes on their websites; and
• the need for more scholarship transparency.

The letter is in response to Mr. Zack and the Section’s assurances that they are addressing what has become a widely-reported call for law school transparency. Over the past year, the Section has had two committees, the Standards Review Committee and the Questionnaire Committee, both tackling the issue of better informing prospective law students. The Questionnaire Committee will recommend changes to the annual questionnaire to the Section’s Council in June. The Standards Review Committee will recommend its changes to the Section’s Council as early as August.

Senator Boxer and her staff are well-informed on the lack of law school transparency and understand the impact it has on our profession. With this understanding, Senator Boxer is in a position to acknowledge that these committees are off to a good start. Committee members have prioritized these issues and have taken input from all sides in formulating their proposals. However, as Senator Boxer’s letter indicates and as we have outlined before, the proposals currently on the table still need work.

Section Committee members need to continue exploring how best to adequately inform prospective students about the significant investment of earning a law degree. There also needs to be substantial pressure on the Council to approve the committee proposals in June and in August. Finally, Council members will also have to determine whether the proposals go far enough in terms of content, access, and auditing. And as we wrote earlier this week, additional Congressional involvement may be appropriate if the ABA doesn’t do its job.

For these reasons we are renewing our call for the Section of Legal Education to establish a new disclosure standard that meets LST’s criteria, while at the same time improving access to (and understanding of) employment and cost information. Continued congressional scrutiny is making it ever more obvious that the public is demanding accountability. We look forward to hearing the ABA’s response.

• Senator Boxer's Letter

The last two months have seen two notable actions concerning oversight of the ABA Section of Legal Education, which accredits law schools and regulates their behavior. At the end of March, California Senator Barbara Boxer put some pressure on the ABA President, Stephen Zack, to ensure that the ABA Section of Legal Education appropriately addresses the lack of quality employment information. The underlying idea: put the ABA on notice that a Senator is watching and that the Section of Legal Education needs to produce results.

Adding to the discussion, the outgoing president of the student bar association at BC Law School, Nate Burris, made public yesterday the creation of a coalition of 55 law school student bar association presidents. The coalition seeks congressional relief for the lack of law school transparency. (The full text of the press release is below. h/t Above the Law.)

Mr. Burris began contacting SBA presidents en masse about a month ago. (The full text of one of these emails, which we received from a tipster, is also below.) In this email, Burris made the following key points:

• Mr. Burris aims to use the support of law school SBA presidents for momentum
• The SBA president coalition planned to present the bill, for introduction, to four U.S. Senators (from Massachusetts and Vermont)
• The bill would require that law schools submit an annual report of employment information to the Department of Education
• The bill would empower the Department of Education to audit these reports

The initial draft of the bill would create a new reporting standard for employment data, with the Department of Education as the collecting body instead of the Section of Legal Education. This standard is the same as the LST Standard, except that it does not indicate who pays the salary, which is now an important distinction given the development of school-funded bridge programs. Perhaps more importantly, it does not protect graduate and employer privacy by separating employers from the salaries they pay.

LST and the SBA president coalition

We’ve spoken at length with Mr. Burris about our thoughts on the bill’s content and timing. Notwithstanding our concerns, which we discuss below, Mr. Burris is committed to improving law school transparency and we look forward to the conversation the proposal will generate. It will add to the collective understanding of the issues and encourage others holding leadership positions to express their comments publicly.

However, as we communicated to Mr. Burris, LST believes the decision to bypass the ABA Section of Legal Education (“Section”) is jumping the gun. While it is both understandable (and correct) to think that the Section has been too slow and too reticent to change, institutional sluggishness is not enough to justify seeking legislation just yet.

These actions are not yet warranted

Passing this legislation would essentially require a congressional determination that both the Section and the Department of Education are incapable of executing the job that Congress previously delegated. Accreditation authority was assigned to the Section by the Department of Education, to which Congress delegated the authority to appropriate accrediting power.

To alter this relationship, Congress would require enough evidence that the Department of Education has failed to adequately oversee the Section and that the Section has failed to adequately regulate law schools. The situation must be such that Congress feels compelled to do more than simply ask the ABA what the status is on increasing transparency. As Senator Boxer’s press release indicated, the political viability of more involvement is presently low.

This is not to say that attempts to get members of Congress involved aren’t a good idea. Certainly, they can exert substantial public pressure; Senator Boxer’s letter of inquiry may be just the beginning of congressional scrutiny. Presenting a bill to the four Senators may result in more investigation and present another opportunity to influence the ongoing conversation, perhaps ensuring that the Section fully addresses the lack of law school transparency sooner rather than later.

But the SBA president coalition is up against a very strong presumption that the Department of Education and the Section are capable of solving the problem once identified, and that they are willing to take the steps that are necessary to fix it. This presumption is derived from the decision to delegate regulatory authority in the first place. Now that the Section has prioritized employment reporting shortcomings, rebutting this presumption before they take final action (or before it becomes clear they are delaying taking final action) is unlikely. It is far too easy for the Section to reiterate what it told Senator Boxer: ‘rest assured, we are on it.’

Still, we are hopeful that the coalition’s proposal will result in further recognition from political leaders, along the lines of what Senator Boxer has already contributed. We also look forward to seeing more from SBA coalition leaders as they explore ways to improve law school transparency.

Shifting the focus

The most productive action at this time will be ensuring that the Council of the Section of Legal Education, which will eventually vote on any Standard 509 reforms proposed by the Standards Review Committee, considers and accepts a standard that adequately improves the quality of employment information. To this end, the coalition leaders should focus their energy on lobbying the Section to solve the issues that the coalition was formed to address. We hope to continue engaging with Mr. Burris and other coalition leaders to rally support for proposals that can do the job, such as the LST Supplemental Proposal.

This is not to say that engaging the Department of Education and Congress for direct action will never be appropriate and more politically viable. Continuous, national attention has unambiguously put the Section of Legal Education on notice that it has inadequately regulated law schools. But the pressure is first on the Section’s Standards Review Committee and next on the Section’s Council to accept the Committee’s new Standard 509 this year.

The Section must adequately remedy the lack of law school transparency if it wants to fulfill its responsibilities, both to the legal profession and under its delegated authority. If the Section falls short of fulfilling these obligations, it will be time to seek governmental reform. The political viability of getting Congress to reconfigure the current regulatory framework will be highest after the Section of Legal Education fails, not while it is in the process of establishing and voting on reforms.

• Proposed Bill (draft)

For prospectives seeking to make an informed decision about attending law school, the need for consumer information extends beyond disclosing post-graduation employment information. They also need to understand how much the degree will cost. Scholarships, grants, fellowships, and stipends (referred to generally as “scholarships”) are increasingly being used to attract stronger students who might otherwise be unwilling to attend a particular program at full cost. Unfortunately, getting honest information about the value of these scholarships can be almost as difficult as figuring out what graduates do after graduation.

The reason for this is that many scholarships are contingent upon achieving a certain level of academic success in law school, either on a semester or yearly basis. And while law schools are quick to point out the number of scholarship recipients or offers made to applicants each year, they are generally silent on how many students retain those scholarships once they’re in school and competing for grades on a curve. For this reason we have submitted a new proposal to the ABA Section of Legal Education. The proposal includes a new Standard [509(d)] and a chart [view here (.xls)].

Merit scholarships with strings attached are nothing new. They’re also not conceptually problematic when a student knows (or should know) what is required to retain the scholarship. Issues arise, however, when the combination of opaque grading curves and conditions bound to that curve obfuscate the meaning of the terms and limit a prospective’s ability to understand the expected value of the scholarship. Even if a scholarship recipient knows what the curve is, the GPA required to retain the scholarship, and that there are others in the class with a similar scholarship, they are still not able to know the real value of the scholarship because they cannot fully appreciate how those facts interrelate. This information is central to computing the price of law school, and the price of a law school is essential to evaluating law schools.

The lack of transparent scholarship retention information is starting to receive more attention: in fact, in addition to our latest proposal (below), it’s also the subject of an article in tomorrow’s New York Times. [Update: online version. The article also links to Professor Jerry Organ's new paper on the topic. Professor Organ is a member of the Questionnaire Committee; we've previously mentioned his important contributions on that front.]

LST’s Proposed Standard 509(d)

(d) A law school must publicly disclose scholarship retention data for all of its students by preparing and posting on its website the attached chart.
 (1) Scholarship retention information shall reflect the scholarship retention data for each student who:
  (a) matriculated in a JD or equivalent degree program at the law school during any of the three previous academic years for full-time students or four previous academic years for part-time students; and
  (b) received a scholarship for all or part of the student’s first year.
 (2) The scholarship retention data must be gathered and disclosed in accordance with the instructions and definitions issued by the Section’s Questionnaire Committee.
 (3) The scholarship retention information about the previous academic year must be posted on the school’s website by October 31 of the following academic year.
 (4) The scholarship retention information posted must remain on the school’s website for at least three years.
 (5) Any scholarship information pertaining to the potential scholarship recipient’s ability to retain the offered scholarship throughout law school must be disclosed to the potential scholarship recipient in any scholarship offer made.
  (a) The terms of the scholarship offer that are relevant to the potential scholarship recipient retaining the scholarship must be disclosed in plain English to the potential scholarship recipient.
  (b) These terms must be accompanied by scholarship retention information about scholarships with the same material retention terms.
 (6) Any scholarship information the law school discloses must be fair, accurate, and not misleading.
 (7) The term scholarship shall mean any scholarship or grant from internal sources (law school or university).

This proposal lays a foundation for putting better information into the hands of prospective law students. Additional definitions and instructions, either as part of an Interpretation to Standard 509(d) or as part of the Questionnaire Committee’s annual questionnaire, will help prevent schools from gaming the requirements and misrepresenting the underlying data.

The Supplemental Chart

The chart referenced in 509(d) [view here (.xls)] aims to show what happens to scholarship recipients during their time in law school. Understanding the chart begins with understanding that, for each anticipated graduation year, the “entering class” is the denominator for the percentage who “began law school with a scholarship or grant.” Every other percentage uses the number of students who “began law school with a scholarship or grant” for an anticipated class year as the denominator. After each academic year passes, a school will then report how many of its original scholarship recipients from the entering class lost their scholarships.

A few other notes:

• The “entering class” covers all students who matriculated at the school during the previous three (FT) or four (PT) years. Students who matriculate at the start of law school do not, however, necessarily all graduate on time or from the same school. Some students drop out, transfer, take a leave of absence, or are otherwise unable to continue their legal education.
• The “entering class” figure will never change, regardless of the number of graduates that leave the school. Likewise, students who transfer to the school will never be included in the “entering class.” It will also not include those who, for example, earned a scholarship due to high 1L grades or a commitment to public service. Even if a student changes her anticipated graduation year, the original “entering class” number will not change. This rule caters to the need to find a manageable way to collect and present complex scholarship retention data.

This restrictive definition carries consequences, of course. First, schools might look worse simply because people leave their school. But because the goal is to help prospectives determine the expected value of their scholarships, those who “lost their scholarship” should include students who actually lost their scholarship, as well as those who could have lost it but for the fact that they transferred, dropped out, or took a leave of absence. Someone “could have lost” their scholarship when the school could have exercised its option to not provide the scholarship.

In other words, if the scholarship recipient would have had too low of a GPA to retain the scholarship, the school cannot claim that it would have exempted the student from the scholarship requirements had the student not decided to drop out of school. We assume the school would have exercised its option if they could have because a school could have provided a scholarship without strings in the first place.

509(d)(5) Disclosures

In addition to disclosure to all prospectives through the Standard 509(d) chart, schools will have to disclose certain information to the prospective students it has offered a scholarship. This includes scholarship retention information about scholarships with the same material terms offered to the prospective. For example, the school would need to disclose the retention rate for previous scholarships with a 3.0 cumulative first-year GPA requirement.

The 509(d)(5) disclosure requirement recognizes that the statistics in the Standard 509(d) chart aggregate different classes of scholarships, and that a scholarship recipient should be especially concerned with how easily previous recipients retained the same class of scholarship.

Going Forward

We have submitted this proposal to the two committees in the ABA Section of Legal Education: the Standards Review Committee and the Questionnaire Committee. In the near future, we will try to convince the Section to adopt a final version of LST’s proposed Standard 509(d).

Please do not hesitate to let us know if you see any unresolved problems with either LST’s proposed Standard 509(d) or the accompanying chart.

• LST Standard 509(d) Chart

We have heard from many law school alumni and current students about problems they encountered regarding how their school reports post-graduation outcomes. Many have alleged intentional acts of deception on the part of their law schools, whether it’s regarding the reporting of their own employment information or the employment of their friends. At the same time, some commentators have accused the ABA Section of Legal Education of lax enforcement concerning violations of the accreditation standards. One way to encourage better enforcement (and better compliance) is to file an official complaint with the Section of Legal Education.

NOTE: We have requested more information from the Consultant on Legal Education, the Accreditation Committee Chair, and representatives of the Section of Legal Ed in Chicago. This post will be updated when we receive a response.

How to file a complaint

For starters, complaints are governed by Rules of Procedure. The complaint form (.doc) is available on this page, which also explains the complaint requirements and process.

A complaint should include a clear and concise description of the allegation and any evidence upon which the allegation is based (with any relevant supporting documentation). [Rule 24(d)3(i).] You must allege a violation of one or more of the accreditation standards, which you can read through here. The complaint must state the timeframe of the alleged lack of compliance (limited to one year from filing), a description of any steps taken to exhaust the law school’s grievance process, and any actions taken by the law school in response to the complaint. [Rule 24(d)3(ii) and (iii).] Any other channels being pursued by the complainant should be disclosed, including legal action. The complainant must also provide a release authorizing the Consultant’s Office to send a copy of the complaint to the dean of the law school.

Any person may bring a complaint alleging noncompliance with the standards; no other harm or damages need to be alleged. The filing of a complaint can lead to an investigation by the Consultant on Legal Education and sanctions by the Accreditation Committee or Council of Legal Education. Per Rule 16 of the Rules of Procedure, sanctions can include monetary penalties, refunds for part or all of the tuition and/or fees paid by students, censure (both private and public), publication of a corrective statement, remedial action, and probation. A school on probation is at risk of being removed from the list of approved law schools.

Allegations that a school has violated or is currently violating one or more standards are serious. Separate from official sanctions of the law school, culpable individuals may be asked to resign or terminated for cause by their school. A school’s reputation may be damaged even if sanctions don’t ultimately rise to the most serious levels. For these reasons we ask that you consider whether the evidence you have is strong enough to warrant an investigation by the ABA. A suspicion that your employment status was misreported, for example, may not be enough without supporting documentation.

You should first contact the school to request that they cease violating the standard prior to filing a complaint with the ABA. An exception to pursuing this route is if you wish to file anonymously, in which case see the discussion below about site evaluation comments.

What actions might qualify as non-compliance?

Of the 52 accreditation standards that currently regulate law school behavior, only one (Standard 509, found in Chapter 5) deals with employment reporting. The seven interpretations of Standard 509, as with all interpretations, carry the same force as the standard itself. This consumer protection standard requires schools to publish certain “basic consumer information” in a “fair and accurate manner reflective of actual practice.” While the accompanying Interpretations only list “employment rates and bar placement statistics” as basic, this list is not exhaustive. You can read more on the current employment reporting requirements here.

Complaints grouped under this standard might fall into two camps. The first are allegations that the school misreported the employment rates or bar placement statistics, focusing on the text of the Interpretation 509-1. Schools are required to report the employment status of each graduate as of February 15th for the second-most recent graduating class on the annual questionnaire.

If you have reason to believe that you or members of your class were miscounted as of that date, despite having reported accurate employment data, and if you can support that belief with documentation such as emails or surveys, then you should consider notifying the school and filing a complaint. Depending on the allegation, this could take sophisticated coordination. You likely need to document a sizable percentage of your classmates’ post-graduation outcomes to show that the reported percentages must have been wrong. For example, a sworn statement from 10% of your class stating they were unemployed as of February 15th would be good evidence that your school’s reported 95% employment is incorrect.

Many recent graduates have contacted us claiming that there was no way the school reported the results of their class accurately. However, it is important to first understand the reporting requirements to see whether the school was just following protocol, as the standards themselves make it very easy to legitimately hide individual outcomes. You may not think that a part-time job waiting tables should qualify you as employed, but it is appropriate under current reporting standards. A violation under Standard 509 would be if the school counted you as employed full-time, or in a JD-preferred or bar admission-required job to U.S. News.

The second camp of violations would be allegations that the basic consumer information provided on a law school’s website or in promotional brochures to law school applicants is misleading and therefore not presented in a “fair or accurate manner reflective of actual practice.” Supporting documentation would necessarily include the publications, and you should describe why they are not reflective of actual practice.

Not willing to file yourself?

One of the Section of Legal Education’s requirements is that complaints will be closed if they are made anonymously, unless the Consultant determines that there are extraordinary circumstances for keeping someone’s identity secret from the school. We understand that graduates may be reluctant to allege noncompliance on behalf of their schools, and that there may be other situations (for example, employees of the school) where someone might be discouraged from whistleblowing if their name will be dragged through the process.

[We have contacted the Consultant for more information on what has counted in the past as extraordinary circumstances, and will update this post when we hear back.]

Some of the complaint procedures may discourage you from filing. For one thing, a complainant has no right within the rules to appeal a decision to close the complaint by the Consultant’s office. A complainant also will not be informed about the proceedings or given access to view the school’s response if one is requested by the Consultant. If the complaint is eventually presented to the Accreditation Committee, there is no appeal process if the Committee sides with the school. And regardless of the outcome, a complainant will only be notified about the stage at which the matter was resolved. From what we can tell all proceedings are closed to the public.

If you have evidence that a school has been in noncompliance and you believe your situation is an extraordinary circumstance, you can contact LST. We will work with you to determine whether the complaint is actionable, and, if appropriate, file the complaint ourselves. NOTE: This does not guarantee that we will file a complaint; it only means that we will review the information to decide if we want to file the complaint on your behalf.

Complaint alternatives

As an alternative to filing a complaint, you can also file a comment as part of the accreditation process. Each ABA-approved law school is recertified once every five years through a process that is taken very seriously by the administration. To conduct the accreditation, the Section of Legal Education sends a delegation of volunteers, often professors, administrators, and judges, to the school as a member of a site evaluation team. The team visits the school to collect facts and gather opinions, including thoughts of employees and students, so that the Accreditation Committee and Council of the Section of Legal Education can evaluate whether the school is in compliance with accreditation standards.

Comments must be submitted at least eight weeks prior to the next site visit, which are conducted during the school year. You can find a draft schedule for all visits up through 2014 on the ABA Section of Legal Education’s website.

Written comments related to current compliance with the Standards for the Approval of Law Schools may be submitted to the Consultant’s Office. The comments should be sent no later than eight weeks prior to the site visit’s beginning date. Comments should be sent to the Deputy Consultant on Legal Education to the American Bar Association, 321 N. Clark Street, Chicago, Illinois, 60654.

Seven law schools will be audited next fall: Arizona, Baylor, Chicago-Kent, Idaho, Missouri-Columbia, Ohio, and Temple. Another twenty are scheduled for next spring. These visits are an excellent time to ask the site evaluation teams to fulfill their responsibilities by taking a hard look at how a particular law school is educating and potentially misleading applicants.

If you have any questions or comments, please feel free to contact us. We plan to begin filing complaints soon.

Senator Boxer has sent LST a statement on the need for law school transparency.

The 509 Subcommittee’s first draft proposal for a revised Standard 509 is a good start. But as we described in our analysis, the proposed revisions are only the first step towards greater transparency. The proposal does not go far enough to disaggregate the current employment information, resulting in a reporting standard that will still struggle to help match prospectives to the law schools that best meet their career objectives.

We have been working on our own proposal, separate from the LST Standard, for a few months now. We have discussed it with key people in the Section of Legal Education, law school administrators, and briefly with NALP’s Executive Director, Jim Leipold. It was born out of discussion at December’s Questionnaire Committee hearing. These conversations have helped shape The LST Proposal into a solution that meets the needs of all interested parties.

The LST Proposal

Our proposal can and should co-exist with the chart proposed by the 509 Subcommittee. Together, the proposals provide prospective students a quick overview of the employment opportunities at various schools while also allowing a more detailed, holistic view for those students who wish to delve deeper. We are hopeful that implementing the two proposals would result in more informed decisions and a more efficient allocation of students to the schools that best meet their career and educational objectives.

The LST Proposal has two core elements. First, each school would report graduate-level data about post-graduation employment outcomes on a “Job Outcome List.” For each graduate, schools would report, as applicable:

• Employment status
• Employer type
• Full-time or part-time
• Required credentials
• Location
• Whether the graduate received special funding
• Job Source

[View the detailed categories on this chart]

These data are already reported to NALP by all but six ABA-approved law schools (St. Louis University, University of Kentucky, Columbia University, and the three law schools in Puerto Rico). The Job Outcome List would be publicly available.

Second, schools would report known salary data for each graduate. Schools also already report these data to NALP. However, unlike the data on the Job Outcome List, the salary data would not be publicly available. Instead, the Section of Legal Education would create a national database of salary data just like the database NALP already has and reports about in Jobs & J.D.s. The database would include all employment data contributed by law schools each year.

The result would be a public, national database of job outcomes and salaries that respects individual and employer privacy desires. Prospective students would use this database for a general idea of lawyer pay in certain locations for certain jobs, as well as an indicator of the short-term economic value recent graduates are attaining with each school’s J.D.

Mechanics of the National Database

Pairing a national salary database with school-by-school, disaggregated employment information would allow prospectives to understand entry-level salaries without identifying the compensation of any individual graduate. To do this, the database would provide salaries for small, though statistically significant, cross-sections of law school graduates. The cross-sections would be created by using the factors that many prospectives consider to be part of their career objectives: employer type, location, and key job characteristics.

For example, for the Class of 2009 graduates, the average starting salary of full-time bar-required jobs in Los Angeles at law firms with 51-100 attorneys was $97,287. The 10th, 25th, 50th, 75th, 90th salary percentiles are, respectively, $75,000, $80,000, $90,000, $95,000, and $145,000. In Atlanta, the average starting salary for the same category is $107,619, and the salaries percentiles are, respectively, $80,000, $90,000, $90,000, $130,000, and $145,000.

Under The LST Proposal, prospectives would be able to match these salaries to a school’s actual placement track record in different places in different jobs. Under the 509 Subcommittee’s current draft, if a school collects fewer than five salary data points for a particular category, schools report no salary information at all. Prospectives remain unaware of how graduates fared because the only information available is that Y graduates obtained jobs with 51-100 attorney law firms, with no indication of location or required job credentials.

In order to understand what these salary percentiles mean to a prospective student considering X school, each school must provide enough disaggregated information to allow prospectives to match outcomes to the national salary database. This connectivity is crucial to an operational national salary database. This is one function that the Job Outcome List would serve.

There are a few ways to design the database, and we are hopeful that the ABA, NALP, LST, and other interested parties can have open discussions about how to best execute this vision. Initially, it is our view that between one and five years of salary data, back-provided by NALP, can be aggregated to create a richer salary dataset. The number of years used would depend on the type of job and location, as salaries have shifted more or less for different cross-categories of employment outcomes. (E.g. New York City 501+ attorney firm salaries have remained relatively stable within at least the last three years.)

Additionally, it is our view that the narrowest salary picture should be provided whenever possible. If enough data exist for 51-100 attorney law firms in Atlanta, city-level figures should be available. If not, the database would provide the next narrowest regional dataset. These higher-level datasets might be Fulton County, Metro Atlanta, Georgia, the South Atlantic (DE, DC, FL, GA, MD, NC, SC, VA, WV), and the United States. The categories could also carve certain locations out of a larger geographical area. For example, one category might be 2-10 attorney law firms in Georgia minus Metro Atlanta. The possibilities hinge only on having large enough datasets. Regardless of whether the narrowest set is available, each higher-level dataset should be associable with each listed job outcome.

Other Advantages of the Employment Lists

The benefits of this proposal do not end with the addition of elaborate, privacy-respecting salary information to the marketplace. After all, the jobs graduates take are often based on more than salaries, so a proposal that aims to help match prospectives to their best fit cannot end with only salary information. To this end, the Job Outcome List will help prospectives understand the various kinds of jobs graduates take at particular law schools. Its components offer various insights into the entry-level market and how each school fits into that market.

Long-term Help

Focusing on a single year of data is dangerous, but an improved standard must start somewhere. The concern is certain to be more pronounced when there is more disaggregated information available for public consumption. The fear that prospectives will pay too much attention to the first year of new data, while grounded in reality, is but a consequence of improved transparency at law schools. The LST Proposal will be best after three or five years. At that point, prospectives would be more able to discern which schools can best meet their individual objectives. And that should be everybody’s goal.

This is our third post in a series of posts (see the first and the second) where we contemplate the 509 Subcommittee’s draft proposal and the facts needed to understand how it would advance transparency at ABA-approved law schools. This post will explain the new proposal and evaluate it using the three criteria we set out in the second post.

The Subcommittee’s Proposal

On March 14th, the Subcommittee released its first draft proposal for a revised standard for the reporting of employment data. David Yellen, dean of Loyola University Chicago School of Law and chair of the Standard 509 Subcommittee, will present this proposal to the Standards Review Committee on Saturday, April 2, 2011 in Chicago. We will provide updates on any changes that come out of the meeting.

The draft proposal has three parts: a memorandum explaining the subcommittee’s operating assumptions and goals, a new Standard 509(b), and a chart that each law school would be required to fill out and post on its website each year.

The Memorandum

In the memorandum, the Subcommittee states that the goal is to “provide more meaningful and consistent employment information to prospective law students . . . [that will] greatly assist prospective students in making informed decisions about whether to go to law school or which law school to attend.” Right away the Subcommittee recognizes that schools already gather a great deal of data, and that it follows that sharing more information with prospective students will require only a small (and, implicitly, justified) burden.

The Subcommittee describes the consumer protection standard, Standard 509, as “a vague standard” that enables schools to provide limited and hard-to-compare information. The fact that reporting practices vary so widely among schools makes it very difficult for prospectives to understand the employment outcomes of a particular set of graduates. What’s more, the Subcommittee continues to recognize that the presentation of information is occasionally misleading. This reflects previous comments made by Dean Yellen.

The memorandum then cabins the problems with the current information into two categories: employment rates and salary information. The Subcommittee establishes two principles regarding the the first category. First, “the percentages disclosed should be based on the entire graduating class, with only those known to be employed being counted as such.” The second principle regards the variety of jobs graduates take, and the problem of providing misleading impressions about the true successes of a school’s graduates. “[T]he best approach is to require schools to disclose more disaggregated data about . . . categories of jobs.” These categories include nonprofessional jobs, part-times jobs, temporary jobs, and jobs funded in part by the school.

Regarding the second category, the Subcommittee recognizes the limited utility of salary medians and the likelihood that readers will misunderstand what the medians refer to and how they are calculated. The Subcommittee proposes that “all salary information clearly indicate the number of respondents and percentage of all graduates included.” This is an important revision that will change the manner in which many schools currently portray salary statistics. For examples of how problematic this can be, check out LST’s data clearinghouse. (The linked example shows a school that reported a median salary of $160,000, despite it being the median for only about 16% of the entire class.)

Proposed Standard 509(b)

The first proposal made by the Subcommittee is as follows:

Standard 509. BASIC CONSUMER INFORMATION
(b) A law school must publicly disclose the employment outcomes of its graduates by preparing and posting on its website the attached chart.
(1) The employment information must be accurate as of February 15th for persons who graduated with a JD or equivalent degree between September 1 two calendar years prior and August 31 one calendar year prior.
(2) The information must be posted on the school’s website by March 31 each year.
(3) The information posted must remain on the school’s website for at least three years, so that at any time, at least three graduating classes’ data is posted.
(4) The information must be gathered and disclosed in accordance with the instructions and definitions issued by the Section’s Questionnaire Committee.
(5) Any additional employment information the law school discloses must be fair, accurate and not misleading.
(A) Any publicly disclosed statistics regarding graduates’ salaries must clearly identify the number of salaries and the percentage of graduating students included.

The proposed Standard 509(b) requires that schools publicly disclose the employment outcomes of the most recent graduating class as true on the first February 15th following graduation. Schools must disclose these outcomes, at minimum, on the “attached chart” by the first March 31st following graduation. It also requires schools to keep the chart on their websites for at least three years. Finally, it adds a catch-all in 509(b)(5) to protect against predatory, opportunistic practices. This specifically includes a solution to misleading median salary practices that some law schools currently use.

The Chart

[View the chart]

The proposed Standard 509(b) “attached chart” aims to exhibit the outcomes of the entire graduating class as of the first February 15th following graduation. The chart disaggregates the current information into smaller categories to illuminate the outcomes graduates achieve at a particular school. The chart is also the first official recognition by an arm of the Section of Legal Education that salary information is in fact “basic consumer information.”

There are two classes of categories on this chart: employment status and employment type. For each category and subcategory, schools must report the percentage of all graduates, rather than of only employed graduates, as well as the raw number of graduates included in the calculation. This decision aims to limit the impact of creative accounting and less than forthright attempts at collecting employment data from graduates.

The employment status class places all graduates into four exhaustive categories: employed, pursuing a graduate degree full-time, unemployed, and employment status unknown.

The chart breaks “employed graduates” into two subcategories. First, this category breaks all employed graduates into four exhaustive kinds of employment: full-time long-term, full-time short-term, part-time long-term, and part-time short-term.

Second, it breaks all employed graduates into exhaustive categories based on the credentials required (or preferred) to do the job: bar passage required, J.D. preferred, other professional, or non-professional. It then further breaks each of those categories into (the same) four exhaustive kinds of employment: full-time long-term, full-time short-term, part-time long-term, and part-time short-term.

The employment type class breaks all employed graduates into six exhaustive categories based on the type of employer: law firms, business & industry, government, public interest, judicial clerkships, and academic. Of those categories, the law firm and judicial clerkships categories are further broken down by type. The law firms are disaggregated by size and the clerkships are disaggregated by level of government (state or federal).

Finally, full time salaries will accompany each category (except solo practitioners) of full-time, employed graduates whenever there are at least five salaries reported in a given category. These salaries will be reported with a 25th, 50th, and 75th percentile, as well as the number of salaries used to create these salary quartiles. There is also a space for schools to report the total number of jobs they funded.

A Good Start, But More To Be Done

The 509 Subcommittee is off to a really strong start in reforming how schools report employment information. It was made clear to us that this is only a preliminary draft, and that the Subcommittee expects more changes will be made. We hope this is the case.

The principles guiding the Subcommittee are sound. It is true that the information must be meaningful, consistent, and help prospectives make informed decisions about whether to, and where to, attend law school. But the execution of these principles still leaves something to be desired. If approved as a new accreditation standard in its current form, the proposal would certainly help prospective students and drastically cut down on misleading statistics. At the same time, it runs the risk of only providing superficial comfort, because it would not help match students to the schools that best meet their career objectives as efficiently as legal education needs.

As we previously outlined, we will use three criteria to assess the draft proposal.

(1) Does it disaggregate the current information?
(2) Does it demonstrate the economic value of a school’s J.D.?
(3) Does disclosure operate on an accelerated schedule?

Does it disaggregate the current information?

This proposal does disaggregate the current information. It helps show the nature of the jobs graduates obtained and with whom the graduates were employed. But as evidenced by comparing this draft proposal to the LST Standard, the vague “employed at 9 months” standard, where “a job is a job,” can be disaggregated to varying degrees. We’ve concluded that this draft does not disaggregate the current information to an adequate degree.

The more disaggregated employment information is, and the more data provided at that degree, the more likely it is that there will be privacy norm concerns. With these norms in mind, there is a legitimate interest in not disclosing all of the employment data that law schools already collect. On the other hand, law schools already collect all of the data needed to help prospectives make informed decisions, so cost concerns are greatly overblown (as the Subcommittee recognizes). As such, the appropriate level of disaggregation must balance privacy norms against the usefulness of additional disaggregation to anybody trying to understand the entry-level market for a school’s graduates.

It is the job of the Section of Legal Education to use its regulatory power to enforce the right balance. The Section must force schools to share the appropriate level of disaggregated information and must not opt to require less useful information because law schools have competitive concerns. The important question thus becomes how much weight the Section of Legal Education should give to schools that believe that more disaggregated information could (i) hurt their recruiting efforts, (ii) cause prospectives to focus too much on the first job in making their law school decision (as opposed to something else the schools think prospectives should focus on), and (iii) cause confusion through information overload.

Among the opportunities for improvement is how well the proposal connects job outcome features together. It does not disaggregate the locations of these jobs and does not show how the job, employer, and location connect for individual graduates. For example, we might be able to tell that 60% of a school’s graduates are working at jobs that require bar passage, but we do not know what percentage of those are working in business & industry. Likewise, we might know that 15% of a school’s graduates work in 2-10 attorney law firms, but we cannot tell what percentage of those graduates are working there as attorneys. This is not merely a theoretical concern– a sizeable percentage of law school graduates work in non-attorney positions in law firms. The decision to disaggregate further directly contravenes the Subcommittee’s principle against providing misleading impressions about the true successes of a school’s graduates.

Part of the reason additional disaggregation is so important is that it would minimize the effect of national rankings on student decision-making by offering a window directly into what graduates shortly after graduating. With this proposal, a prospective’s choice might still hinge on what a school ranks each year in U.S. News rather than on how well a school can help a student achieve her goals. Prospectives need clarity about how a school fits into the legal hiring market.

After all, the Subcommittee’s stated goal is to help prospectives make “informed decisions about whether to go to law school or which law school to attend.” The proposed solution is only satisfactory insofar that the goal is to differentiate between schools using percentage differences in broad, albeit more disaggregated, categories. It will still be too difficult to know the challenges graduates face for achieving their career objectives, which usually include a combination of location, employer type, and required credentials. Without sufficient granularity, neither will prospectives as easily understand a school’s placement niches. All together, prospectives will still struggle to understand schools’ unique placement abilities.

Another issue with the Subcommittee’s method of disaggregation is that it actually creates new gaps in the information (though not to a debilitating extent) and thus an incentive for creative accounting. One of the purposes of disaggregating the nine-month employment rate is to limit how much schools hide employment outcomes. Unnecessary gaps undermine this purpose.

The total number of graduates in each subcategory, taken together, should equal the total number in the parent category. For example, the total number of graduates who are employed, unemployed, pursuing a graduate degree, or whose employment statues are unknown should equal the total number of graduates in the graduating class because the categories are exhaustive.

The unknown status category is very important for identifying gaps in the employment status data. However, an unknown category is missing from all other exhaustive groups except the group for type of law firms. The employment type category, required credentials subcategory, judicial clerkships subcategory, and the full time and part time (and corresponding long and short term) subcategories all need an unknown field so that the numbers in the subcategories all equal the parent category’s total number.

Helping prospectives understand where data gaps exist encourages them to ask the right questions and serves to limit false impressions due to extrapolating outcomes from unrepresentative segments of the graduating class. Unfortunately, allowing schools to report graduates as “unknown” in any category incentivizes schools to avoid learning or researching employment outcomes. However, it is more important that the gaps created by non-reporting graduates are readily identifiable. As such, all exhaustive categories and subcategories need to account for each graduate.

Does it demonstrate the economic value of a school’s J.D.?

It is a huge step forward for the Subcommittee to recognize salary information as “basic consumer information.” As of right now, the only standardized, school-specific salary information is courtesy of U.S. News. Until this year, even U.S. News salary information was too opaque.

The Subcommittee’s proposal does a decent job with highlighting what new graduates make and, accordingly, demonstrates some of the economic value of each school’s J.D. This new salary information would allow prospective students to roughly understand how well graduates can service their debts immediately after law school. For the Class of 2009, the average graduate had $98,055 of law school debt, which translates to about a $1200/month loan payment.

While the Subcommittee’s approach is useful and likely the best way for schools to report school-specific salary outcomes without using job-specific salary data, it is not the approach we think the Subcommittee should take. A better way would be to leverage the reported salary data of all law schools together the way NALP does in its annual Job’s and J.D.’s. Certainly, if prospectives knew about this publication, which costs non-members $90, they could use it to have a better understanding of entry-level salaries for law school graduates. But there is currently no way to bridge the gap between this salary information and an individual school’s graduates, and the Subcommittee’s proposal does not help on that front, so it is limitedly useful for those trying to decide which law school to attend.

The aforementioned lack of connectivity between employers, job credentials, and job location makes understanding how the new salary information impacts them – particularly for loan payments – very difficult. For example, a $160,000 starting salary for a new associate grows differently in New York City compared to Houston due to salary compression in years two through seven. Additionally, $70,000 in New York City does not go as far as $70,000 in Philadelphia, Raleigh, or Nashville. The geographic impact on the ease of loan repayment cannot be understated. Even if a prospective has the Job’s and J.D.’s book, that information can only take them so far because its salary breakdowns are very specific (e.g., attorneys in 2-10 person law firms in X city). Nothing in the new standard or chart helps answer these important questions.

There is a separate concern about whether each category would have meaningful salary information associated with it. For example, 10 may work at small firms, with only four reporting. In this case, the four salaries do not get reported and thus do not serve any use. They are simply swept away. However, if these four salaries were added to a national salary database, those four become 40 or even 400, and the result is meaningful salary information about jobs that wouldn’t otherwise have salary information. Unfortunately, this resource cannot be utilized on a school-by-school basis without more disaggregation. In our next post we will explain our proposal for doing this in depth.

Does disclosure operate on an accelerated schedule?

Yes. In striking this balance between cost concerns and the need for timely information about the most recent graduating class, the Subcommittee has paved the way for significant improvements beginning as early as next year. At the Questionnaire Committee hearing in December, law school administrators expressed concern that requiring schools to report information too soon would be too high of a burden given cost constraints. But by limiting the Standard 509 requirements to only data that schools submit to NALP in February/March, the Subcommittee erases these concerns. Even small career services staff will be able to comply with the standard provided they already report to NALP, which nearly every ABA-approved law school does. Given that collection methods are now mostly electronic (through Symplicity or other user-entry databases), assembling and posting the data according to the proposed Standard 509(b) would take very few work hours and limited financial resources beyond what schools already allocate voluntarily.

Concluding Thoughts

The goal of a revised Standard 509(b) must be to help students make informed decisions about which (if any) school best meets their career objectives. While a good start, we think that, as currently conceived, the Subcommittee’s proposal will fail to adequately achieve this basic goal.

We ask that each member of the Committee imagine herself as a prospective student trying to choose a school to invest thousands of hours and dollars into. Each member must then think about how soundly she can act after analyzing employment information reported according to the new standard, and consider how well she actually understands the school’s ability to help her achieve her career objectives. We suspect that this thought experiment would leave each member uncomfortably uncertain. This uncertainty, at a minimum, should be addressed through a non-theoretical exploration of the standard’s implications. Before accepting a new standard, the Standards Review Committee should compare a few schools using real employment information presented as it would be under the proposed revisions.

An improved Standard 509 has the ability to wage an important battle against the influence of U.S. News on the decision-making of prospective law students. But without sufficient disaggregation of the current employment information, the effects can only be minimal. Under the current proposal, it is still too easy to imagine a prospective student choosing the #55 ranked school located on the east coast over the #81 ranked school on the west coast because she does not know, for example, what to make of the schools’ minute differences in percentage employed in mid-sized firms as it pertains to her goals of working out west in a mid-sized firm. Without adequate information to dissuade her, she might come to the head-scratching conclusion that #55 must be better because it is ranked higher. This is bound to worsen now that there are 45 more schools ranked on a national scale.

Each year, the Section of Legal Education makes an effort to minimize the effect of national rankings. We are sure that almost every law school administrator would agree with the Section’s sentiments, and revising Standard 509 is the chance to show that these are not empty words. We look forward to working with the Subcommittee to improve this first draft.

This is our second post in a series of posts (see the first) where we contemplate the 509 Subcommittee’s proposal and the facts needed to understand how it would advance transparency at ABA-approved law schools. This post provides three criteria for us to use to judge the ABA’s actions. In our next post, we will evaluate the new proposal using these criteria.

Whatever standard the Standards Review Committee and Questionnaire Committee together adopt, it must:

(1) Disaggregate the current information
(2) Demonstrate the economic value of a school’s J.D.
(3) Operate on an accelerated schedule

(1) Disaggregate the current information

 
The most serious handicap of the current reporting standards is that the standards allow outcomes to be hidden in aggregate form. For prospectives seeking to make an informed decision, and law schools seeking to fulfill their educational responsibilities, the new standard must provide an accurate picture of the entry-level job market for each school. To do this, any new standard must characterize the jobs graduates obtain beyond “a job is a job.” This includes the nature of the jobs graduates obtain, with whom the graduates are employed, and the locations of these jobs. Gaps in the information also must be clearly visible to limit prospectives from extrapolating from unrepresentative segments of the graduating class.
 
The best way to achieve this is by requiring graduate-level detail, just like NALP has been collecting for years. This allows prospective students to know the challenges they face for achieving their educational and career objectives, which will help them maximize the value of their time spent in law school. The granularity also respects school regionality and encourages schools to develop their placement niches. Whether this niche is in a particular region or city, a field of law, or a sector, this feature publicizes each school’s unique placement ability. Displaying where all graduates go post-graduation can help match students to the right programs, minimizing the effect of national rankings on student decision-making. The choice then becomes less about what a school ranks each year in U.S. News and more about how each school can help a student achieve her goals. If it is clear where a school fits into the legal hiring market, schools will be encouraged to adapt and innovate, and may even be able to reduce costs.

This does not mean that law schools must share how much each individual graduate makes at her first job, as we have done with the LST Standard. Rather, law schools just need to provide enough graduate-level detail to enable prospectives to make a meaningful connection between the post-graduate outcomes for a given school’s graduates and the regional market rates for those jobs.
 

(2) Demonstrate the economic value of a school’s J.D.

 
While disaggregating the current information into graduate-level detail allows for rough estimates of economic value, the ABA does not currently consider salaries to be basic consumer information. It is time for the ABA to recognize the importance of starting salary as basic consumer information. Some prospectives come to law school straight from undergrad with low opportunity costs, and others change careers or work first, but almost all will eventually pay an enormous amount of money for the privilege to earn a J.D. It is difficult to separate the question of “how much will I make?” from “how much will my monthly loan payments be right after I graduate?” Likewise, it is difficult to think about the salary a graduate earns separate from where that graduate lives and works. New salary information must be presented in a way that allows prospective students to understand how graduates begin earning the income they need to juggle loan payments, living expenses, and everything else a new member of the legal profession must pay for.
 
It is clear that a graduate’s starting salary is only a part of the economic value a graduate can derive from the degree, and that many graduates (notably solo practitioners) may see a sharp upward trend in their earning salary over the first five to ten years. However, entry-level salaries are a good place to start, and the least costly time to assemble a comprehensive picture of a graduating class. The Bureau of Labor Statistics provides salary information for lawyers, but lawyers represent only a portion, even if a large one, of law school graduates. The important question is the value of the law degree itself. Between 60% and 70% of all 2009 law school graduates had jobs, as of February 15, 2010, that required a J.D. Of those that did not, some will eventually find work as an attorney. Likewise, some of the graduates who work as lawyers after law school will soon leave the profession. None of this warrants hiding information about post-graduate outcomes. Career trajectories are hard to predict, but they all necessarily include the first job.
 

(3) Operate on an accelerated schedule

 
The data and information reported on the annual questionnaire and on law school websites must be published in a timely manner. The 2009–2010 questionnaire was due October 31, 2010. This included employment information about the class of 2009, which was finalized on February 15, 2010, and will not be published anywhere until after the admissions cycle for the Prospective Class of 2014 has just about concluded. The Class of 2009 information will not appear in the Official Guide until after the Class of 2010 data has been assembled and reported to NALP. The ABA must publish this information sooner, along with other consumer information as it becomes available.

There are no reasons why law schools cannot either submit employment data to the ABA or provide employment information on their websites by the end of March, each year, for the most recent graduating class. According to Jim Leipold, Executive Director of NALP, data straggles into NALP from February 15th through March 15th, but by the end of that period almost every school has reported all of their employment data to NALP. This data is fresh in everyone’s minds and can be readily provided to the consumer at low costs to career services staff.

In the old days, there would be good reason why prospectives needed to wait to see this data, because the submissions to NALP by law schools would be by individual paper forms. Simply put, times have changed. 90% of law schools submit the data in an electronic format, downloaded from whatever system the school uses to survey graduates. And although schools already have the Class of 2010 data accessible, to our knowledge no school has posted any 2010 employment information on their website.

While the problems with the current employment information (see our white paper for more detail) are separate from the terrible job market, the present job market makes the current reporting schedules unquestionably unacceptable. Regardless of whether job placement for a given year was good or bad, prospectives should still be able to see the full picture. But when the available information is so outdated that it differs greatly from current placement trends (as evidenced by the new NLJ 250 statistics), not providing up-to-date information to consumers grossly undermines the obligations law schools have to their students and to the legal profession. This is particularly true when the information has already been collected and can be disclosed to the consumer with relative ease.

These three evaluation criteria were originally communicated to the Questionnaire Committee and Dean David Yellen (Chair of the 509 Subcommittee) at the December 2010 Questionnaire Committee hearing by LST’s Executive Director.