New York University School of Law has decided to continue withholding valuable employment data from prospective students. After sending NYU a third request to publish its NALP report, we finally received a reply from Dean Ricky Revesz:
Thank you for your note. I expect you are aware that, since the end of last year, we have added a substantial amount of employment data to the NYU Law website. If there is more information that would be suitable and helpful for us to provide, we are happy to consider doing that. For example, we are now looking into posting data of the type found in Table 12 of the NALP form (Source of Job by Employer Type), since that would likely be of interest to prospective and current students. Please let me know if there is additional information that you think would be helpful for us to publish.
Our letter was quite clear about what would be helpful: NYU needs to publish its NALP report. It was a short letter and referenced the NALP report five times. (Read it here.) There is no other way to read any of our three requests and it’s insulting that Dean Revesz would feign ignorance while referencing the very report we requested.
We responded to Dean Revesz, repeating that we believe the entire NALP report should be published and noting what information NYU has not released (along with the sections of the NALP report that contain this information):
- Full-time and part-time employment by required/preferred credentials (“FT/PT Jobs”)
- Classification of business jobs (“Business Jobs”)
- Position held in a law firm (“Type of Law Firm Job”)
- When the job was obtained (“Timing of Job Offer”)
- Salaries by job type (“Employment Status Known” and “Employment Categories”)
- Salaries by location (“Jobs Taken by Region” and “Location of Jobs”)
- Salaries by firm size (“Size of Firm”)
Dean Revesz’s response to our last communication was disappointing:
As I indicated in my last note to you, we will continue to add data to the area of our website that contains employment statistics.
We must read this to say that NYU has no plans to publish the additional information we have requested, even though it is of value to both the legal profession and law school applicants. The data NYU indicated it was “looking into” publishing 14 months after collection–the source of job by employer type–has not been added to NYU’s website despite more than another month passing. The immense intellectual, technological, and financial resources of NYU have proven unequal to the task of posting an 8×7 table.
(To see the data NYU knowingly withholds, check out our reconstruction of its NALP report in our data clearinghouse. Fields shown in dark gray represent information NYU possess but has not released.)
“Having trouble knowing what to believe? We have a proposal for Paul Campos: come audit our numbers. We’ll show you a list of all NLJ 250 firms to which we sent associates in 2010 and 2011. Pick a reasonably sized sample from that group, and compare them to firm-verifiable data. Then let us, and the world, know what you find.” – NYU’s Rebuttal
Professor Paul Campos called in to question NYU’s biglaw placement rate, citing a discrepancy between the numbers reported by NYU and by the National Law Journal. NYU’s response was clear:
Come at me, bro!
Something about NYU’s response (I’m a 2008 grad) just didn’t sit right with me. It wasn’t the acerbic tone the rebuttal took. It was that Law School Transparency had already requested NYU be more open about its job placement rates. LST actually made that request twice, and we have renewed that request again today.
Like all-but-six law schools, NYU has in its possession, right now, a NALP report with detailed job placement statistics for the class of 2010. This report contains a wealth of information, including the size of firms students went to work at; salary information for a multitude of categories; if the jobs are full time or part time, permanent or temporary; what states they are in; if the job at a law firm is as a lawyer, a clerk, or a paralegal; if students found the jobs through OCI, a job posting, or went back to work at a pre-law school employer. It paints a very detailed picture of what happened to the class of 2010, but it’s a picture that NYU has decided to not let anyone else see.
That’s what makes NYU’s response to Professor Campos so strange.
NYU professes its openness, its honesty, its transparency. With one hand it extends an offer to verify that things are as it says they are, but with the other hand it folds up the numbers and locks them away. “Come audit our books…. But not those books!”
Having trouble knowing what to believe? We have a proposal for New York University: disclose your numbers. You’ll publish the employment data collected by NALP for the class of 2010 (and for 2011 when you receive that report this summer). Then let us, and the world, know what you already know.
In this month’s National Jurist, the magazine’s editors used the data from our Live Transparency Index to grade how transparent each law school website is. While we do not encourage or approve of ranking or grading schools based on the number of criteria met, because some criteria are more important than others, we did provide National Jurist with a spreadsheet of the data to ease their workload. These are the same data that were available on the Live Index as of the end of February 2012. As such, some schools have updated their websites since that time.
In other news, another school provided LST with its class of 2010 NALP report: St. Mary’s University. Over the next few weeks, we will be encouraging a letter writing campaign to convince other schools to provide us with their class of 2010 NALP reports. In the meantime, we are busy inputing the data from all NALP
Update: We were alerted that Indiana–Bloomington had made its NALP report available online prior to this story’s publication. We’ve made this correction throughout this story.
Update 2: We apologize to the University of Utah S.J. Quinney College of Law. On February 10th, the school provided LST its NALP report, but we did not include it in this story.
Update 3: To see more schools that have since complied, see here.
On December 14, 2011, we wrote all ABA-approved law school deans to request the class of 2010 NALP report that each school received in June 2011. We are pleased to announce that we have obtained 32 34 of these reports (17.3% of ABA-approved law schools and 17.8% of schools with NALP reports). 29 of the 32 34 law schools sent a report to LST. The other three four were pulled from school websites.
We asked for these reports to help prospective law students find the law schools that best meet their career objectives. Together, these reports provide prospectives access to timely, thorough, and comparable employment information. They make LST’s website an even more valuable source of information for prospective law students. We hope to soon add the employment data contained in the NALP reports to our data clearinghouse. In the meantime, these reports are available here. In addition, the list below links to each school’s NALP report.
Schools Providing the Class of 2010 NALP Report to LST (30):
Non-Responding Schools with accessible NALP Reports (4):
We want to make a special note that six law schools do not submit employment data to NALP and therefore do not have these forms: Notre Dame, Pepperdine, St. Louis, and the three law schools in Puerto Rico. This does not mean that these schools do not have ample employment data, however. Rather, these are the only schools who do not have a prepared form in front of them that can quite easily be disclosed to prospective law students.
Interestingly, the vast majority of schools providing LST with the 2010 NALP reports were public institutions. This may be because these schools interpreted our request as an open records request. Two schools (Wisconsin and UNC) explicitly treated our request in this way.
We received a handful of responses expressly declining to provide LST with the NALP report. Consistent with past communications with law schools, a number of schools indicated that they had meaningful employment information already available on their websites. In almost every case, this was (and remains) false.
In addition, there were two “no” responses that stood out. The full text of the responses is below.
First, Ave Maria argued that its NALP report does not provide meaningful information. Dean Milhizer claimed, “Our school is small with a unique mission, and our employment outcomes are reflective of this.” He suggested that additional information would be needed to assist prospective students. He did not, however, suggest what kind of information would be useful. The school’s website contains little information on what graduates found for work, with much of that information serving to mislead applicants.
Second, Chapman University argued that its report was confidential. Dean Campbell’s response is very misleading. It is true that NALP promises that it will not share the graduate-level data or the reports generated from those data with anyone except the law school. But schools are under no such obligation, either contractually or (for the NALP reports) legally. If it was required to keep the data confidential, Chapman could not provide these employment statistics on its website. Instead, the NALP reports only remain confidential because law schools decline to share them with those who would find the information most useful. Fortunately for prospective students, at least 32 33 schools disagree that the NALP reports are confidential.
Overall, as the Live Transparency Index shows, schools are increasingly sharing more employment information. But the vast majority of law schools still leave critical gaps in their presentation of employment information – gaps which the NALP reports would fill. These 32 34 law schools have demonstrated leadership that is sorely lacking at other law schools. While these schools still need to choose how to share employment information on their websites, they understand the importance of providing free access to comparable information now. Our hope is that these schools pave the way for changes at other schools, many of whom are still acting as if their applicants do not deserve access to comparable consumer information.
Ave Maria School of Law:
Dear Mr. McEntee and Mr. Lynch:
I applaud your efforts to assist prospective law students in obtaining reliable information about the law schools of interest to them. However, Ave Maria School of Law does not believe that its Class of 2010 Summary Report from NALP will provide meaningful information about our school to prospective students. Our school is small with a unique mission, and our employment outcomes are reflective of this. In our judgment, the Summary Report does not provide sufficient information about the types of positions obtained by our 2010 graduates, and so to release the report in a vacuum without additional information would not be of assistance to prospective students.
This year, as in years past, AMSL will comply with NALP’s guidelines on reporting employment outcomes for the Class of 2011, and we will be participating in the ABA’s new annual survey of these outcomes.
Eugene R. Milhizer
President and Dean
Dear Law School Transparency,
You have requested that our School of Law send you the otherwise confidential report that we supply to NALP. We have years of experience with NALP, and on that basis, we know that the information we send will be appropriately treated, consistent with good ethics and all applicable federal and state laws. Regrettably, we do not have that kind of basis with your organization. Accordingly, we think it is most appropriate to continue to keep our submission to NALP confidential.
Nevertheless, our School of Law maintains a very informative website, and we post a great deal of data on our graduates’ employment there. That information is available to the public, and we invite you to consult this source if you would like.
This morning, we sent a letter to all ABA-approved law schools asking that they provide us class of 2010 employment information so that we may expand our data clearinghouse. Our goal is to provide thorough and comparable employment information to prospective law students. While some law schools are improving the quality of information they share, it is critical that people be able to compare law schools through a standardized presentation standard.
It is true that the ABA Section of Legal Education and Admissions to the Bar has taken important first steps towards reducing the provision of misleading information by law schools. However, these steps have failed to include critical information about the class of 2010, including the rate of graduates employed in legal jobs and the rate of graduates employed in full-time jobs. The section underestimated how prospective law students, pundits, and elected officials would react, despite mounting evidence of widespread consumer-disoriented behavior at law schools and within the section.
We hope that schools share our sense of urgency and help us put comparable employment information into the hands of consumers.
Originally published in the National Law Journal.
Critics calling for law school reform are rousing an old discussion about problems with legal education. Recently, their focus has been on the provision of misleading job placement statistics. People are tired of law schools’ dishonest tactics, a sentiment that grows as the number of examples of fraud and corruption increases. Furthermore, they are beginning to understand the negative externalities caused by students unwisely choosing to attend law school, both to the legal profession and elsewhere.
The main problem with the employment information stems from the American Bar Association Section of Legal Education and Admissions to the Bar, which includes any job in its basic employment rate. Law schools truthfully advertise rates above 90% because they report employment data according to the section’s standard. Nevertheless, these advertisements mislead prospective law students when coupled with two popular yet distorted consumer beliefs: that lawyering is a lucrative profession and that the rates reflect legal jobs.
Law schools are aware of these distortions, but they have no pecuniary incentive to tear down the information asymmetry that protects the legal employment rate. Ever the optimists, prospective law students do not discover the realities of a school’s job placement until too late. Until recently, structural problems with employment information have been the profession’s dirty little secret.
The number of affected graduates has grown during the past few years, but the problem is not unique to the post-2009 job market. Since the turn of the century, just two-thirds of all ABA-approved law school graduates obtained jobs requiring bar passage within nine months of graduation. Neither the ABA-Law Schools Admissions Council Official Guide to ABA Approved Law Schools nor the vast majority of law school advertising materials inform consumers about this reality. Meanwhile, tuition and graduate debt are on the rise, salaries are deflating and the legal market is increasingly more saturated. Calls for consumer protection, even if logically independent of these additional facts, are common sense for a profession with high ethical standards.
In response to public pressure, the section asserted that it would pass reforms to reduce the provision of misleading employment information. This would have prevented consumers from being led to believe that the basic employment rate was the legal employment rate. Instead, the section is taking steps that ensure that next year’s applicants will actually have even less information. The section reasons that this is a transition year, more information will be available in the future, and that the short-term loss of information quality is worth the section reasserting its accreditation authority. This reasoning is accompanied by a misplaced concern for whether the definitions used to categorize job data are adequately defined. In finalizing these steps, the section is breaching its responsibilities to the profession.
For years, the section has had the ability to share how many graduates were finding full-time legal positions from individual law schools. The section collects these data in its annual questionnaire, which asks schools to report each graduate’s employment status (employed, unemployed, pursuing another degree), employer type (law firm, government etc.), and other job characteristics such as whether a job requires bar passage or is full time.
One might ask why the section has never published job characteristics data in the Official Guide, or why law schools rarely share this information in their own materials. These are important questions. But the more pressing question is why the section is trying so hard to come up with justifications for not publishing the data for next year’s incoming class.
On Sept. 23, the section’s questionnaire committee will finalize the 2011 questionnaire, which asks about the class of 2010. Additional reforms are slated for 2012. If nothing changes, the section will collect fewer job characteristics data than it has collected in prior years. Apparently, whether a job requires bar passage or only prefers a J.D., or whether a job is full- or part-time, are now too obscure to define without many more meetings. These definitions have been developed by the National Association for Law Placement and have been integrated into the questionnaire for many years. While not perfect, the definitions adequately meet consumer needs. Changes will always be necessary to reflect law school practices and market shifts, but feigning lack of consensus over commonly accepted terms should trouble even the most optimistic observer.
It is odd that, under the auspice of improving information, the section is actively reducing the amount of useful information available this year. This move will have ramifications beyond the questionnaire. Among the schools that report these important statistics on their Web sites and to U.S. News & World Report, some will jump at the chance not to share how well (or how poorly) the class of 2010 fared in finding legal jobs. These schools can hold up the section’s misplaced skepticism as their justification. Prospective law students deserve more from the law schools, but they can’t get it just by asking nicely.
If the section is truly interested in fulfilling its obligations to the legal profession and as an accrediting agency, it needs to consider whether a do-nothing policy is the appropriate course of action given the events of the past two years. The stakes are too high for the section to hide behind imagined concerns and continue to let law schools pull the wool over prospective law students’ eyes.
The Council of the Section of Legal Education announced today that it would move forward in collecting graduate-level data from law schools. As we reported yesterday, the Section and NALP will collaborate as to limit the negative effects of this policy on NALP’s annual studies of the entry-level hiring market.
TORONTO, Aug. 6, 2011 — The Council of the American Bar Association Section of Legal Education and Admissions to the Bar is announcing that it will move forward in collecting detailed job placement data from law schools, and will hold schools accountable for the completeness and accuracy of that data. The decision comes at the end of the section’s council meeting held at the ABA Annual Meeting in Toronto.
As the federally recognized law school accreditor, the council has the ability to require law schools to meet specific standards for accreditation.
“Our regulatory function puts us in the best position to be able to collect data from law schools and ensure that it is reliable,” said section chair Chief Justice Christine Durham of the Utah Supreme Court. “We will begin requiring that law schools report job placement data directly to us,” she said. Previously, law schools voluntarily provided job placement information to a trade association, the National Association for Law Placement. The section and NALP have agreed to collaborate going forward.
During the past year, the ABA Section of Legal Education’s Questionnaire Committee engaged in an extensive effort to respond to concerns that current data was either inaccurate, insufficient or both. Beginning next month, the annual law school questionnaire will require schools to report more specific information than ever before, including employment status, types and locations. The questionnaire will ask these questions, among others:
- Is the graduate employed or unemployed?
- Is the graduate’s employment long-term or short-term?
- Is the job funded by the law school or university?
- Does the graduate work for a law firm, a business or in government?
“The section is committed to providing this data so that applicants, students and the public can make informed career decisions,” said Bucky Askew, legal education consultant to the American Bar Association. The section will report the information in the ABA-LSAC Official Guide to ABA Approved Law Schools.
With nearly 400,000 members, the American Bar Association is the largest voluntary professional membership organization in the world. As the national voice of the legal profession, the ABA works to improve the administration of justice, promotes programs that assist lawyers and judges in their work, accredits law schools, provides continuing legal education, and works to build public understanding around the world of the importance of the rule of law.
As we discussed Wednesday, the relationship between the ABA Section of Legal Education and NALP has been rocky over the last week. Members from both organizations made statements to the press and bystanders asked whether this would be the end of NALP. Concerned with that possibility, we sent a letter to the Section and to NALP urging the two to compromise.
Now, the ABA Journal reports that NALP and the Section of Legal Education have decided to cooperate:
On Friday, both groups spoke of their long history of collaboration on data collection and production, particularly on employment and placement figures, according to Hulett “Bucky” Askew, the ABA’s consultant on legal education. Both NALP and the council spoke of the desire to continue to work together moving forward.
Although Askew did not confirm whether an agreement had been met on the methodology the section would use to collect data directly from the schools, a prior point of contention for NALP, he did say that the two groups will continue to meet and discuss ways to address the needs of all parties.
This is reassuring news for the Section of Legal Education, NALP, prospective law students, and the legal profession as a whole. We will monitor the situation closely and report on the compromise when more details are known.
On July 27th, after over a year of assessing different methods of improving transparency, the ABA Section of Legal Education announced important changes to its plan for collecting employment data. In a memorandum sent to all law school deans and career services officers, Bucky Askew (Consultant on Legal Education) and Dean Art Gaudio (Chair of the Questionnaire Committee) revealed that the Section would begin collecting graduate-level employment data from the law schools, as opposed to merely collecting data in the aggregate (such as the percent of a class employed in a job or the percent who passed the bar). The memorandum is attached to this post below. Once finalized, the Section of Legal Education would become responsible for collecting hundreds of thousands of data points each year, a task which has historically been undertaken by NALP.
Currently, the Section of Legal Education does not collect granular employment data from law schools. Instead, schools must only report very basic information about the entire graduating class on the annual questionnaire (read more here), a practice which has permitted widespread misunderstandings about the nature of the entry-level hiring market. NALP, on the other hand, annually administers its own detailed survey to gather data about individual graduates.
Despite the voluntary nature of reporting these data to NALP, an overwhelming percentage of ABA-approved law schools (192 out of 199) take the time to do so. From this sizable dataset, NALP cleanses the data for discrepancies, generates private reports for each school to assess its own performance, and creates general reports about the state of entry-level legal hiring. Schools then have the option of using the private NALP-generated reports to respond to the U.S. News survey and ABA Annual Questionnaire. Schools can (but don’t) choose to release these reports to members of the public, specifically to prospective law students. NALP does not release any school-specific information due to agreements it has with its member schools.
As we reported in June, the Section of Legal Education has already made important strides, in its adoption of a new policy that will increase the amount of information available about each school’s entire graduating class. We said:
[T]his solution will provide prospectives a more thorough and accurate window into the placement opportunities at various law schools. However, we do not think this solution goes far enough with consideration to our goal of helping prospective law students find the schools that best meet their individual career objectives. The adopted solution does not provide enough graduate-level detail for those seeking to make a meaningful connection between the post-graduate outcomes for a given school’s graduates and the job characteristics, including salary, required credentials, and location. It is still too difficult to connect these dots.
While the Section of Legal Education has not decided to share the graduate-level data with consumers, a finalized policy would indicate that the Section understands the importance of the underlying data, not only for understanding the aggregate information schools report, but for enabling auditing and safeguarding against fraud.
Section officials are aware that LST supports the Section in its decision to collect graduate-level data from the law schools. The Section has failed to adequately regulate law schools on certain issues for too long, and this will contribute to the Section fulfilling its regulatory function. Accreditation is at its core a matter of consumer protection and the rules governing accreditation accordingly need to be sufficiently robust to protect consumers. The Section owes this duty to the profession and to those who wish to enter it. It is not in the profession’s or prospective students’ interest to have anybody misled or ripped off.
In a letter to Mr. Askew and Dean Gaudio, NALP’s executive director, Jim Leipold, expressed in no uncertain terms his anger over the Section of Legal Education decision to begin collecting graduate-level employment data in this manner.
We object to this action on several grounds, including the fact it will actually lead to LESS transparency and information about the entry-level legal employment market and not more, and the fact that it is an action that is contrary to all of the public conversations about this issue that have taken place among the ABA, NALP, the law schools, and the public over the last year and a half.
. . .
One of the chief harms caused by this action is that it will require a dual reporting burden by the law schools, who now will be asked to report individual student record level employment data to both the ABA and NALP.
. . .
Worst, we fear, is that if schools are required to separately report employment outcomes to the ABA, there is a great risk that many of them will no longer report their data to NALP. This will inevitably lead to the reduction in the amount of information we have about the entry-level legal employment process, and will have the long-term effect of producing less transparency about the legal job market and not more.
These are important concerns. NALP has provided useful information about the entry-level legal market for 37 years, and there is certainly risk that some schools will stop reporting to NALP due to the dual burden. Such big-picture analyses of the health of the legal industry are useful for schools and employers trying to gauge larger trends in hiring shifts, but they cannot be expected to replace the work of an accrediting agency. If the Section of Legal Education is finally deciding to fulfill its accreditation responsibilities fully, this decision should be given a certain level of deference. In other words, if only one group receives the underlying data, it should be the accrediting body and not the third-party relying on voluntary reporting (which itself is enhanced by privacy agreements that make the data inaccessible to those who need it).
However, this is a false dichotomy. There is no need for the ABA Section of Legal Education and NALP to clash; the two can co-exist seamlessly. NALP collects an enormous number of data each year and not only is the process well thought out, but the definitions are useful, coherent, and authoritative. NALP also already collects almost all of the data the Section can reasonably desire to collect itself. In the interim minor differences will exist, such as conflicting definitions of what jobs qualify as short-term employment, but there is little reason to believe that NALP would not be willing to negotiate the terms if the two groups reached a mutually beneficial understanding. However, this requires reopening the discussion and mending the relationship with NALP quickly so that the annual questionnaire may go out at the end of this month.
Recognizing the important and historically complimentary roles of both the Section of Legal Education and NALP, we believe that a compromise in the collection of employment data is both achievable and desirable. Our suggestions for reaching such a compromise utilize the following premises:
– That the Section of Legal Education actually does wish to collect employment data at graduate-level detail.
– That it is important for the Section of Legal Education to fulfill its accreditation obligations, which encompass the collection of employment data at graduate-level detail, so as to limit fraud and enable auditing where such auditing is shown to be necessary.
– That NALP already collects these data and more.
– That at least some law schools will not participate in NALP’s survey under the Section’s proposed changes, because they believe doing so would be too costly.
– That if fewer law schools participate in NALP’s voluntary survey, it will damage NALP’s ability to provide systemic employment information to schools, the legal profession, and prospective law students.
– That, if in the end only one of the Section of Legal Education and NALP can collect employment data, it should be the Section of Legal Education.
– That both the Section of Legal Education and the ABA should respect and value NALP’s longstanding service to the profession and engage in dialogue with NALP’s leadership.
– That, if possible, NALP’s function should be preserved.
– That the Section of Legal Education can fulfill its accreditation responsibilities by using the questions and definitions NALP has fashioned over the years, and does not need to reinvent the wheel.
It’s important to remember what the real fight has been about when discussing law school transparency: the optimal level of information. Schools already collect enough data to more than adequately inform prospective law students. Yet, these data remain private and inaccessible to those who genuinely need quality information. As such, the success of any reforms hinges on the quality of information that follows after schools report data. But this is not the controversy before us today. This is a clash over who can collect and access the underlying data.
As we said above, this clash is unnecessary. The Section of Legal Education and NALP need to work together, not deride each other in the press and behind closed doors. NALP should communicate a willingness to cede the final say on post-graduation outcome surveys, and in exchange continue to gain access to the data.
The simplest (and also cheapest) way to achieve this is for NALP and Section of Legal Education to use the same survey. Under this model, NALP would use the Section of Legal Education’s survey that happens to be based on NALP’s survey. Each year, NALP and the Section can discuss changes, but the Section would have the final say. This does not constitute outsourcing a regulatory function to NALP, something the Council of the Section of Legal Education legitimately fears doing, but it does recognize and utilize NALP’s great work over the past 37 years. NALP’s role would diminish only as far as the Section does not defer to its institutional expertise in making changes to the survey.
The Section has the power to put NALP out of the employment statistics business, and it should not wield this power irresponsibly. But it should also not forget that it has a responsibility to the profession and to those who wish to enter into it, and that some of this responsibility can be shouldered by NALP without outsourcing its regulatory function. Adopting NALP’s survey and inviting NALP to help change it in the future is the right thing to do given the obvious pressure to better regulate law schools. There would still be details to work out. For instance, the Section would need to invest significant resources into technology (especially to ensure that NALP and the Section do not end up with different data) and staff. Similarly, NALP would need to share its wisdom and processes for cleansing the employment data.
This is an important problem that needs to be solved immediately. NALP contributes a great deal to the legal profession, and the Section wants to enhance its own contribution. Through this particular compromise, the two groups can maximize contribution and continue a long-standing relationship.