On Saturday (3/17), the Council of the ABA Section of Legal Education, will hold an open session to discuss a variety of topics. (Agenda here.) Most importantly, the Council has an action item on Standard 509, also known as the ABA’s consumer information standard. This standard sets the baseline requirements and operating principles for law school disclosure of consumer information.
We have submitted a memo for the Council’s consideration. In this memo, we urge the Council to adopt the proposed Standard 509 with several small changes. To view the memo, click here for a PDF or view the entire text below the fold. We expect the Council to pass the proposed Standard 509. We hope that it consider integrating some of the changes before the vote, and that the other recommendations receive due consideration when the Council finalizes the two charts and charts’ directions.
LST’s executive director’s statement on behalf of LST
This Memorandum regards the Standard Review Committee’s proposed Standard 509. We urge the Council to adopt the proposed Standard 509 with several changes.
I. Standard 509(a)
“Violations of these obligations may result in sanctions under Rule 16” [emphasis added]
The addition of this sentence has no substantive effect on law schools because the ABA Standards already empower the ABA to sanction law schools for violations of Standard 509. Rather, this addition merely indicates to curious, concerned parties that the ABA takes its consumer protection responsibility seriously. While we do not suggest a strong mandatory minimum sanction, we do suggest that the standard require (rather than simply permit) remedial actions in the event of incomplete, inaccurate, or misleading information. Further, we recommend that the disciplinary body enact a default requirement that the school post notice of the corrective action in all media where the information has been posted, which the body can decline to issue with reasonable cause.
While determining whether to sanction the school, we recommend that the disciplinary body be required to take into consideration the severity of the bad information, the efforts to notify prospective students of the error once realized, and the speed with which such information is audited or corrected once discovered. This creates an opportunity for law schools to mitigate prospective sanctions by self-reporting its errors/misdeeds to consumers. For corrections, time is of the essence unless the school elects an external audit. In those cases, we recommend that the Standards require that the school stop posting the information it has elected to audit and instead post notice of the audit. The latter notice is particularly relevant when the school removes consumer information from the website otherwise required by Standard 509.
II. Standard 509(d)
The proposed Standard 509(d)(1) provides a March 31 deadline; we believe it is important that the Council not change this date. This date provides current applicants information before most deposit deadlines and provides the next year’s crop of applicants ample time to use the most recent employment information in deciding where to submit applications. Extending the date beyond March 31 would significantly decrease the utility of the information, as well as the likelihood that applicants would have time to react to the information before they must decide whether and where to attend law school the following fall.
Later dates, especially after June, may appear to be attractive alternatives. After all, NALP returns employment data cleansed and neatly organized in June. But while NALP works with schools to update data from February-June, the 509(d) Chart is not an immutable chart. It is therefore appropriate to require schools to post the employment data shortly after collection, and then encourage them to correct the data after the third-party NALP analyzes them. It would be odd to base the disclosure date on a school’s voluntary compliance with an external, non-regulatory party’s research process.
III. Standard 509(e)
We propose changing the language to (suggestion in red):
A law school shall publicly disclose on its website, in the form designated by the Council, its conditional scholarship retention data by October 31. A law school shall also distribute data to all applicants being offered conditional scholarships at the time the scholarship offer is extended.
IV. Employment Outcome Data Presentation Chart
It is important to emphasize that the Council should vote, as the Standards Review Committee has proposed, to require law schools to provide school-specific salary data, although aspects of the school-specific data are rightfully questioned. School-specific data are less representative than the state-specific datasets. Moreover, when only 50% of graduates report a salary, granular categories like “Employed in a Law Firm of 2-10 Attorneys” are unlikely to have sufficient school-specific data to meet the five salary minimum. These concerns motivated the Questionnaire Committee to aggregate salary data by city and state for various employment sectors.
Nevertheless, the school-specific salary data’s shortcomings do not rise to such a level where it would be appropriate to permit schools to withhold such statistics. These statistics add an element to a prospective’s understanding that the state-level statistics do not. The decision not to include school-specific data would be an enormous step backwards for the proposed Standard 509. In the absence of school-specific data, the best indicator of a school’s value would be the private sector salary quartiles and public sector median salary provided by U.S. News (for a fee) coupled with whatever the schools choose to provide on their own. Additionally, it would reduce the incentive schools have to collect salary data from their graduates, a necessary task, as the state salary information relies upon school-collected data.
We do not mean to suggest that the recommended changes, if implemented, would resolve all of the problems with the current collection and presentation of school-specific salary data. The Council should remain open to ways to improve response rates and the accuracy of school-specific numbers. By divorcing the chart’s specifics from Standard 509, the Standards Review Committee impliedly recognized the importance of such flexibility.
We recommend the following changes to the chart:
- The chart’s title includes a misnomer. It should say “Class of —- Employment Report as of Feb. 15, ” instead of “at 9 months after graduation.” The current title is unnecessarily confusing to readers and may cause future confusion among career services staffs. In this same vein, the directions should clearly reemphasize what Standard 509(d)(1) already says.
- For the sake of consistency, “Employed – Job Category Undeterminable” should say “Employed – Job Category Unknown.” This would be consistent with “Employment Status Unknown” and “Employer Type Unknown.”
- It is important that schools fill the “# Full Time Reporting Salary” column cells even when the number is less than five, and even though the salary information in such instances will be blank. The directions should also include an express prohibition on providing salary information when N is fewer than 5.
- It is important that the “Business & Industry” row receive the same treatment as “Law Firms” and “Judicial Clerkships.” The school would indicate what employers comprise the business category. Schools already have these data because the NALP survey requires it, even if the ABA does not. The same applies to “Government” and “Public Interest.” A number of schools have already decided to voluntarily disclose these subcategories because it provides meaningful information about the kinds of jobs their graduates take.
- It is important that the “Employment Location” chart also include salary information for the three states. This would permit comparisons to the aggregate figures to see how the school stacks up to other schools with similar geographic concentrations.
For the Council’s Consideration in Devising Instructions and Definitions:
When deciding upon the instructions and definitions to be used by schools in complying with Standard 509, we recommend that the Council consider a number of important questions. What, if any, should the outer date limit be on whether a graduate unemployed as of February 15 is “Unemployed – Start Date Deferred” or “Unemployed – Not Seeking” and “Unemployed – Seeking?” For example, if a May 2012 graduate obtains a clerkship beginning in June 2013, will this count? What if it starts in June 2014?
- Related: If the June 2014 clerkship recipient does not count as “Unemployed – Start Date Deferred,” how should this graduate count?
- Related: If somebody is deferred in a job requiring bar passage, but has a non-professional job as of February 15, will the school get to decide how to count the graduate? Or will the Council mandate how this graduate is reported?
- Related: Will the Council, as NALP has done in the past, treat law firm deferrals the same as other deferrals?
The instructions should make it unambiguous that the “Employment Status” categories are exhaustive, with the total of all columns equaling the total number of graduates. The same is true for “Employment Type” but with the total equaling the sum of the “Employed – ” (excluding Unemployed – Start Date Deferred). The same is also true for any subcategories of “Employment Type,” as the Standards Review Committee has done with the law firm subcategory and the “Unknown [Size]” row.
These gaps also exist in the full-time/part-time long-term/short-term matrices. An “Unknown” cell should be added under “Full Time” and “Part Time” (parallel to the “Long” and “Short” columns) as well as after those matrices (parallel to the “Full Time” and “Part Time” columns), with this last cell including any graduate for whom job duration or job hours is unknown. The end result would be seven (instead of four) categories for each row used to describe job duration and job hours. Through our research for LST’s Transparency Index, we have discovered that this is a common trick among law schools and this is the only way to make it impossible to use.
Without clarifying the instructions in this manner, the method of disaggregation creates gaps in the information and creates an incentive for creative accounting. One of the purposes of disaggregating the nine-month employment rate is to limit how easily schools can hide unattractive employment outcomes—whatever each school deems these to be. Unnecessary gaps undermine this purpose.
Helping prospectives understand where data gaps exist encourages them to ask the right questions and serves to limit false impressions that follow from extrapolating outcomes from unrepresentative segments of the graduating class. Unfortunately, allowing schools to report graduates as “unknown” incentivizes schools to avoid learning or researching employment outcomes. However, it is more important that the gaps be readily identifiable.
V. Scholarship Data Presentation Chart
The purpose of Standard 509(e) and the accompanying chart is to allow scholarship recipients to understand the expected value of their scholarships. To this end, the form provides past data to show how difficult it is to keep scholarships at a particular school. It is critical that the form be tailored to this purpose, although simplicity must be an important factor in creating the chart.
The proposed chart’s third column, “# Whose Conditional Scholarships Have Since Been Reduced or Eliminated,” provides a quick, useful numerator for the probability in the expected value calculation. However, its plain language fails to capture certain students that the numerator needs. If a student transfers, drops out, or takes a leave of absence, that student no longer has a scholarship. Leaving these ex-students out of the calculation will tend to mislead the reader.
We have identified two basic solutions to this problem. First, the Council could choose to change the column to, “# Conditional Scholarship Recipients with Unchanged Scholarships.” This has many downsides without additional nuance, however. For one, it overstates the difficulty of keeping a scholarship when top students transfer. Likewise, it overstates the difficulty when a scholarship is only one year instead of renewable for three.
Instead, we recommend maintaining the original column, but capturing lost scholarships through a definitional trick. Those who “lost their scholarship” would include students who actually lost their scholarship, as well as those who could have lost it but for the fact that they transferred, dropped out, or took a leave of absence. Someone “could have lost” their scholarship when the school could have exercised its option to not provide the scholarship. This keeps the focus squarely on the difficulty of keeping a scholarship.
LST has identified a few other problems as to the quality of the information provided in the chart. Notably—much like the ABA’s original employment reporting standard—it aggregates different classes of scholarships. It is not uncommon for law schools to offer scholarships of different classes based on monetary amount and GPA/class-standing requirements. This very real tactic, while not inherently bad, is an opportunity to game this chart. For example, a school could offer a $500 conditional scholarship, with a minimum class standing of the top 95%, to all students in an entering class who did not receive another scholarship. Like any merit scholarship, this will benefit the school’s U.S. News ranking via the expenditures segment of the methodology, but now it will also make all other scholarships appear easier to keep. With numerous unfortunate and real possibilities like this, it seems to us that, at the very minimum, the Council should commission the Questionnaire Committee to collect broader data to allow for the Council to make informed decisions in later years as to the importance of expanding the data schools must share with scholarship recipients. It is more desirable to closely monitor potential developments in merit scholarships through real data, especially because merit scholarships are one important reason the cost of legal education has risen well beyond inflation, than to wait for schools to make changes that may harm future, unknowing applicants.
VI. Necessary Interpretations
Expansion of Interpretation 501-2:
We encourage the Council to incorporate the following suggestions:
- “Complete” [as used in Standard 509(a)] means accounting for the whole graduating class at any point a school uses statistics to provide information about the graduating class. If a school collects data about Statistic Z for 80% of the class, the school shall indicate the 20% non-response rate with respect to Statistic Z.
- If a school claims or implies that information (e.g. a list of employers hiring its 20__ graduates) is representative, and it is not, that information is incomplete, inaccurate, and misleading.
- When publishing consumer information, a school shall consider its applicants’ knowledge, beliefs, and cultural influences. These factors affect how consumers understand information. Schools shall apprise themselves of how their consumers think and make publication decisions with consumer thinking in mind.
- Any “rate,” such as a nine-month employment rate, must unambiguously identify how it was calculated.
- Arguments that incomplete, inaccurate, or misleading information would not change consumer behavior are irrelevant to an analysis of whether Standard 509 has been violated. The fact that the information was published at all implies that it matters to consumers.
Change to Interpretation 509-3:
- Int. 509-3: “percentage of all graduates” instead of “percentage of graduates”
VII. Additional Comments and Suggestions
(1) School-Funded Jobs
The Council should consider creating a new category to parallel the other employed categories under “Employment Status.” “Employed – University-Funded” would include short-term jobs funded directly or indirectly by the law school, as well as long-term jobs where the duration is finite. Under the current system, a graduate employed in a short-term job funded by the school almost always counts as a job requiring bar passage. Under the new system these graduates would no longer be treated this way. Schools already collect sufficient data to do this for the class of 2010 and 2011. Similarly, and more importantly, the same could be done on the “Employment Type” chart, with “School-funded” operating parallel to “Law Firm,” Judicial Clerkships,” etc. Some schools have already begun to do this so that they do not mislead their prospective students.
(2) Long-Term Jobs
Starting with the class of 2011, NALP splits the “long term” job category into two groups. The first group includes jobs with a known or fixed duration, e.g., 1 year, 2 years, or project-based. The second group includes jobs that do not have a known end-point. The ABA should adopt this method.
The term “academia” is very misleading. It implies that graduates are working as academics at colleges and universities, or as law professors, when in reality it usually means working as a research assistant, in the library, or in the admissions office. We suggest that the Council eliminate this category, disaggregate this category (like “Law Firms” or “Clerkships”), or coin a new term, and that the Standards Review Committee work with the Questionnaire Committee, NALP, U.S. News, and LST to determine which option(s) would be most appropriate. If eliminated, many outcomes fall into “public interest,” while the rest naturally qualify as “business & industry.”
(4) Interpretation 509-1
The term “significant” is unclear and weak. Listing any course that isn’t being offered and hasn’t been offered for years should be a violation. Schools have access to their own course catalogs, and under no circumstance should they be allowed to advertise any courses that do not offer.
(5) Regulatory Barriers to More Efficient Law Schools
It has become apparent that legal education has gotten away from legal educators. There are almost 200 schools vying to be Harvard-like think tanks. The vast majority of these schools set tuition prices in a distorted market, with rates loosely based on a school’s U.S. News ranking and geographic location. This pricing model, which relies on student loans and dwindling credibility, will not survive. The Council should acknowledge this reality and ensure that the ABA standards do not stand in the way of schools needing to substantially change how they deliver a quality legal education.
At some point, as more and more graduates question their own ability to practice law upon graduation and more clients refuse to pay for their services, we must do more than idly theorize on changes to the current model. We believe the Council’s first step should be to conduct an inquiry into how the accreditation standards (a) functionally prevent low-cost alternatives and (b) could be adapted to allow other models to emerge. For too long, cost considerations have been absent from reform discussions. The Council should seize the chance to better legal education for the sake of the profession and society at-large.
In November, we asked that Jeffrey Lewis, dean emeritus and professor of law at Saint Louis University School of Law, create a special subcommittee to review regulatory barriers preventing law schools from adapting low-cost models. To date, plans for such a committee have not been announced. It is critical that the Council ask Dean Lewis to create this committee today, and that the Council urges the new committee to act quickly and thoroughly. If the answers do not come quickly from legal educators, the result will be that educators end up forfeiting their right to control the changes. And if the answers have to come from elsewhere, unbreaking the broken law school model will be as painful as it is necessary.
• • •
I want to thank the Council for the opportunity to submit this Memorandum. As an organization that will follow these developments closely, Law School Transparency is committed to assisting the Council in its continued efforts to improve law school regulation and oversight.
Kyle P. McEntee, Esq.
Law School Transparency
March 17, 2012